Course Workshopping: Library Instruction for Econometrics

Last week I wrote about the course workshopping my team did over the summer, to revamp how we teach particular library instruction sessions. In that post, I promised to write about how that course workshopping changed what I was doing with ECON 203: Econometrics, a mid-level, required course for all economics majors at Wellesley.


Here’s what the old instruction session looked like:

I went to each section of ECON 203 for 15 minutes. I was a total talking head. “Here are the three aspects of the group projects I can help with. Here’s the research guide. Here’s how to sign up for a group meeting.” There was literally nothing about this instruction session that was pedagogically effective.

  • There were no learning objectives.
  • There was no opportunity for interaction between me and the students in the class session.
  • When the students came to see me after class, they were ridiculously confused about what they needed.

Gag.


Workshopping prep message to my colleagues

Background: this is a required economics course. Most majors take it in the their second or third year. There are typically 3 sections each semester. The instructional technologist meets with them to teach them how to use Stata and, depending on the class, IPUMS. I meet with most sections each semester for a 15-minute intro to finding data and literature, in their classroom. There are computers in the classroom, but the students generally don’t have them fired up and ready to go when class starts and I’m hesitant to change that class culture. Mostly, we review the guide.

Typical post-session questions from students:
  • Literature:  does whether gambling is legalized in the state when one turns 18 affects an individual’s decision to go to college? JSTOR and EconLit results are mostly about athletic gambling or gambling among college students.
  • Policy: finding state by state marginal income tax rates by year – hoping to find something like this.
  • Policy: history of standardized testing across states by dates
  • Data: crime rates by state; compulsory education by state; how to find other panel data by state
  • Policy, literature, data: looking at effect of US immigration policies on the real wage of Americans, before and after of 1965 lift of the national origins quota system and want suggestions on other recent immigration policy changes or incident similar to the Mariel Boatlift on the Miami Labor Market through which there was a significant increase in immigrants of specific origin in a specific region, in order to do a cross-comparison among different states.
What I need help with:
 
I’m wondering if there is something more useful I could do in that time.
  • Would it be useful to take a sample question and work through it in that time? If so, what question might be good?
  • Where do I focus my time most effectively in this class: finding data, finding policy, or literature research?
  • Would it be useful to give the students a pre-meeting assignment? If so, what would be on it?

Results of the Workshopping Session (in photo form, click to embiggen)

Notes from course workshopping session
Notes from course workshopping session

My instructional technology colleague also workshopped her two sessions with this course. Then we set up a meeting with the faculty members teaching this fall to see if we could try something different in all three of our sessions. One of the ideas we pitched was that the students would all get an article ahead of the class session and read it, then in class, they would think-pair-share about:

  • what data set is used?
  • what variables are used from the data set?
  • what is the policy change or moment in time when something changes?

One of the faculty suggested we use this awesome article by Douglas Almond about the fetal origins hypothesis and the 1918 influenza pandemic in the U.S.

Not only were the faculty members were on board with the idea, they also agreed that 15 minutes wouldn’t be long enough, so they shifted my visit to their lab session.


Here’s what the new instruction session looked like:

Both instructors invited me to attend their lab session instead of a class session; the students were in a different frame of mind to work in that session.

In the first faculty member’s lab, we agreed to send the students this synopsis of the article ahead of time. Then in class, she led all of us through an in-depth discussion about how you would construct a study to answer the question: if your mother had the flu while you were in utero, would your life outcomes be worse than people whose mothers didn’t have the flu? Then we handed out a printout of the data section of the Almond article (p. 683-686) to the students and I led them in a discussion of the questions we’d identified: what data sets are in use; and what variables does the author use?

In the second faculty member’s lab, I flew solo (the instructor didn’t attend – eek!) The students read this blog post ahead of time, and read the data section of the article in lab. In pairs, the students answered the following questions:

  • what is the research question in plain English?
  • what data sets does Almond use for his analysis?
  • what are the x (independent) andy (dependent) variables?
  • why didn’t Almond just find out whether the moms of the babies had the flu?
  • what’s the strength of the his two methods of analysis?

As a large group, we closely examined the data section of the article , identifying  names of variables, years when those variables were available, etc. We went into IPUMS-USA and selected the samples and variables Almond used. Then we walked through how to create an extract, change it to Stata format, and download it.


Outcomes

What a difference doing actual instruction makes! I’ve met with more than 75% of the small groups so far this semester, and every group has come to me significantly more prepared to work with the concepts in their project.

  • They understand the difference between the data they are finding in IPUMS and the data they are finding that reflects a policy change.
  • They are conversant with how to look at, read, and understand the variables and samples they identify in IPUMS.
  • They are still asking me questions about how to build their queries in Stata (I happily pass them to my instructional technology counterpart for that) but we can have really good conversations about how they might want to logically consider their question.

In short, I’m thrilled with the changes I see in the students after changing this instruction session up. I can’t wait until the end of the semester to talk with the faculty members to see what their impressions of the group projects is. I hope to use that feedback when talking with the faculty member who’s teaching the class in the spring, to see if she’ll be on board as well.

Learning to Identify Sources – A Short Lesson Plan

A couple of days ago, on a list I follow, someone posted a link to “Just what am I looking at?” from the Distant Librarian. The person was wondering if anyone had any ideas about how to “unflatten” the online world for students. It reminded me of an article Barbara Fister wrote called Rebundling the Unbundled Article. A recent Project Information Literacy report – How Freshmen Conduct Research Once They Enter College – find that “many freshmen were unfamiliar with the formats of scholarly publications before entering college.” When you add in the challenge of everything appearing on a computer screen, things get complicated.

I don't get it

In response to the message, I shared the following in response (my version here is significantly edited, because blog post = editing, while email reply = less editing.) This is by no means a unique or novel exercise, but sometimes reading someone else’s take on common activity will spur you to try something a little differently next time. With that, enjoy!


One of my colleagues demonstrated this activity in her interview. I have adapted it for several 100-level first-year writing courses, where students struggle to understand how various sources differ.

Identifying Types of Sources – A Hands-On Exercise

materials needed:

  • 4-5 copies each of a scholarly journal, a trade publication, and a popular magazine* related to the course topic (depending on class size, you may need more)
  • handout describing the different types of sources (see any of these web pages for inspiration)
  • in-class worksheets
  • access to computers
  • whiteboard

Students brainstorm on the whiteboard about the kinds of sources they’ve used in the past; the list usually includes web pages, books, scholarly articles, movies, and magazines. I explain that in the realm of “articles” there’s a lot more diversity than meets the eye, and learning to discern what’s what isn’t a something we spring forth from the womb knowing how to do.

The students break into groups of 3. Each group gets one of each type of print source and a worksheet. Each student gets a copy of the handout describing different types of sources. The students examine each item and look for clues to help them identify its type. We typically work for about 15 minutes on this part. I encourage the students to fill out the worksheet.

After  15 minutes, each group reports out on one item. They tell the class what they thought it was, and why. The rest of the class agrees or disagrees with them, and provides reasons why. Each of the report-outs takes about 2 minutes. Students tend to do pretty well identifying the scholarly journal and popular magazine, but struggle a lot more with the trade publications. They tend to have a lot of good conversations during the reporting-out.

At that point, the students always appear to have a lot of confidence about what they’ve just done (with good reason!), so I throw them the following questions:

  1. Where do you normally find materials for your papers? (Cue the chorus of “online!”)
  2. How do you figure out what kinds of materials you’re working with online?

There’s usually a moment of hesitation(unbearably long, no matter how short it might actually be), and then someone will say “could we try using the clues we just used for the print sources?”

And so we do! In small groups, the students use our discovery layer to identify an item on a topic of interest. With an item on-screen, they answer the following question::

Is this a scholarly, popular, or trade source? List at least 3 items from your handout which help you make that claim. Are there clues not on the handout which also help you determine what kind of source this is? List those too.

We come back together as a group to report out again, and it’s AMAZING how quickly and easily the students are able to accurately describe the what their online items are. The clues from the handout are helpful, but I’m always surprised at how many students use the search facets in our discovery layer to limit items to different types of sources as well without any prompting from me or their professor. In classes where we haven’t done this kind of exercise, the students rarely notice the discovery layer facets (even when they might be really useful.)


Outcomes

I did this exercise twice in classes last spring. I ended up meeting with many of the students in the classes after our instruction session as they tackled their research projects. All of those students were able to describe what they’d already found in the language they learned in class, and were able to say, “I’m looking for more popular sources” or “The scholarly sources I found were too complex for me to understand with my current level of knowledge. How can I find scholarly sources that aren’t so complex?” I count both of these reactions as anecdotal evidence that the students learned in our sessions. The fact that the professor reported back to me that the students asked her fewer source-type questions AND wrote papers that fulfilled her requirements around types of sources is the other piece of evidence I use to point to this as a useful exercise to adopt in classes where students struggle with understanding how to contextualize sources they find online.

Thumbs up!
Thumbs up!

* This can be really hard in some academic libraries! For one class I actually went to a bookstore and purchased 3 popular environmental sciences/spirituality magazines last spring because we just didn’t have anything like that in our collection – or our staff lounge! – in print.

I’m Megan Brooks, Librarian at Wellesley College. This is How I Work.

In the vein of the popular LifeHacker series of posts, I thought I’d take a stab at describing how I work.

Location: Metrowest Boston
Current Gig: Director of the Research Services team, a group within Research & Instructional Support, which itself is part of Library & Technology Services at Wellesley College
One word that best describes how you work: Intuitively
Current mobile device: iPhone 5S
Current computer: at work, a 13″ MacBook Pro; at home, a 13″ MacBook Air

What apps/software/tools can’t you live without? Why?

At work, I’m a diehard Google Apps fan. We use Google Apps for Education at MPOW, and frankly, without it I’d be lost. Nearly everything we do is now someplace in the Googlesphere; groups for discussions, drive for shared and private files, and sites for organizing everything.

For my personal life, I am a big fan of Feedly on iOS and the web for following blogs; TweetBot for Twitter on iOS, IFTTT for moving things from here to there automatically, and Instagram/Flickr for photo sharing. Plus CrashPlan to backup my computer. Yay CrashPlan!

What’s your workspace setup like?

At home, I sit in an IKEA Poang rocking chair, but at work, I’ve got a much more ergonomically-friendly setup (plus I got in on the StandStand Kickstarter, so my options are only going to increase in the ergonomic department going forward.)

my office setup
My Office Setup

This view shows my inspiration wall – feminist postcards, a small monthly calendar, Dead Feminists broadsides, a painting I did at one of those painting places and dog postcards. I’ve also got an IKEA lamp (for when the overhead lights aggravate my migraines), a couple of chairs for when I meet with colleagues or students, my office phone, and an analog clock. This shows my computer setup a bit better:

computer setup
Computer setup

Here we’ve got the MacBook Pro (normally locked up when I’m not in the office), an external keyboard, mouse, and raised monitor, with a dual screen setup and a super old mousepad showing off Our Lady of Guadalupe.

As you can tell, I keep my desk really clear of paperwork, books, etc. I can’t abide having a cluttered working surface, and tend to clean up every night before I go home. People get a little weirded out by that, because I guess it’s not a typical librarian trait? Oh well. I’m a librarian, and it’s my trait, therefore it’s a librarian trait.

What’s your best time-saving shortcut/life hack?

I have one that gets me out the door in the morning. I set out all my outfits for the week on Sunday (including socks, undergarments, belts, accessories, etc.) While this might feel very elementary school, it actually helped me solve that problem of staring into my closet in the morning wondering what on earth to wear. I’m not a morning person (AT ALL) so being able to grab an outfit the night before from the pile – an outfit I know looks good because I put it together mid-afternoon on Sunday, when I was well-rested – saves me time and mental anguish. My brain works better in the evening, so my morning brain trusts the decisions my evening brain made. Win-win for everyone!

What’s your favorite to-do list manager?

I hate to-do lists and to-do list managers (my brain does NOT work that way), but I am giving Trello a whirl, to see if it works any better than my whiteboard or post-its. So far, I’m a bit happier than lukewarm on it, which for me is saying something.

Besides your phone and computer, what gadget can’t you live without and why?

At work, I need a good whiteboard and whiteboard marker. At home, I’m rather fond of my Kindle Paperwhite. But I could easily live without either of those.

What everyday thing are you better at than everyone else? What’s your secret?

Okay, this is weird and totally NOT related to work at all. I’m really good at being able to tell if something is level or not. You know how some people have perfect pitch? I’m like that with pictures hanging on the wall. No need to grab a level if I’m around. I have no idea what my secret is for that – I can just tell.

Excuse me, I have to go adjust the painting on the living room wall for a moment.

(I’m not kidding.) (It’s level now.)

What do you listen to while you work?

The sweet sounds of silence. For real: I don’t like listening to music when I work because I find it distracting.

What are you currently reading?

At home, I’m reading Who Fears Death by Nnedi Okorafor. She’s a Nigerian-American speculative fiction writer and she’s amazing and I can’t wait to read everything else she has written ever. I recently read Akata Witch, and got hooked.

For work, I’m just checked out Putting the User First: 30 Strategies for Transforming Library Services by Courtney Greene McDonald, and hope to do a quick read-through over the weekend.

Are you more of an introvert or an extrovert?

Both personally and professionally, I’ve spent most of my life as an extrovert – I do get energy from being with people – but with strong introvert tendencies – I need my downtime to recharge.

What’s your sleep routine like?

I try to go to bed before 11pm every night, although lights out can happen anytime from 9pm to 2am. My alarms go off at 6:30, 6:40, 6:50, and 7:00 (yes, I have 4 alarms in the morning. I did mention I’m not a morning person.) I’m usually up by 7:10 during the week. On weekends I usually sleep until 8 or 9am.

Fill in the blank: I’d love to see _________ answer these same questions.

Twitter librarians! You know who you are.

What’s the best advice you’ve ever received?

These are my two favorite pieces of advice, the second in direct conversation with the first.

From my dad: If you can’t find the time to do something right, when will you find the time to do it over?

From a good therapist: Make sure you know if it’s something that needs to be done in the first place.

Edited to add: props to Michael Perry for doing this a couple of weeks ago and inspiring me to do the same!

When Tried and True Doesn’t Work Any Longer: Workshopping Library Instruction

There comes a time in many instruction librarians’ careers when all of a sudden, it’s really obvious that what they’ve been doing in a class just isn’t cutting it any longer.

  • Maybe there’s a new faculty member teaching the class.
  • Maybe professor changes what they’re teaching in the class.
  • Or maybe, just maybe, everyone is bored with the same old, same old (this is particularly true for those mid-level, required courses in a discipline – I’m looking at YOU, econometrics.)

This past summer, the team I’m on decided to tackle this problem in a head-on, low-risk, manageable way. Everyone picked one course they regularly taught that they wanted to revamp in some fashion, and the rest of the team helped them think through how to change it.

The courses we workshopped each week this summer ranged across the disciplines and across course levels:

  • 100-level first-year writing course focusing on Wellesley College
  • 200-level required econometrics course (librarian)
  • 200-level required econometrics course (instructional technologist)
  • 300-level French course on La Belle Epoque (in French)
  • 300-level history seminar on ethnic and religious violence

You’ll notice we had no science courses, for a good reason: our science librarian was on leave for the summer.

The format of each session worked the same way. Each person sent out material ahead of time, including:

  • course description
  • background on the course
  • description of how they have been involved in the course, learning outcomes, etc.
  • assignments, handouts, guides, or other supporting materials
  • description of “what I need help with” – which ranged from figuring out how to talk with an instructor about changing an assignment to dealing with a class where people in the class might be complete novices or subject experts to using extremely limited class time differently to basing an instruction session solely on discussion

Each workshopping meeting lasted an hour. The person who was “on” that week gave a brief overview of what was in the material she sent out and provided more context than what would fit in email. The conversation went from there. The “on” person always had free rein to say, “this conversation is straying and I need us to focus over here for a while.” (Because we are a chatty, idea-filled group, this happened more than once.) Typically, the “on” person and one other person in the room took notes or wrote on the whiteboard, so there was a record of all the wonderfully smart, intelligent, and 100% feasible ways to revamp the instruction.

In one of my later posts, I’m going to write about how I revamped instruction for econometrics, focusing on how this workshopping meeting gave me a different kind of confidence when I spoke with the faculty members teaching the course and asked if we could please please PLEASE do things differently because I was being swallowed alive by supporting the students after our session.

(Interestingly enough, the teaching and learning center at MPOW is hosting a series of similar lunchtime meetings for faculty this year, based on the Knotty Problem Roundtable developed by Mary Rigsby and Suzanne Sumner at the University of Mary Washington. I’m eager to try that method out with this team in the spring, focusing on different courses. The barrier to entry seems lower than what we did last summer – although the outcomes might be less dramatic as well.)

If you’re in New England, it looks like NELIG’s December meeting is going to be exactly this kind of thing! If you don’t have a group of colleagues at your workplace who you can do this kind of workshopping with, consider applying to present/get feedback from the wonderful NELIG community! Deadline for that is tomorrow, November 7, so get on it!

In Which I Finally Pass a MOOC

You know what’s kind of sad? I work at the first liberal arts college to join EdX, and I’m a serial MOOC (massive open online course) dropout. I’ve tried; I honestly have. I’ve signed up for several MOOCs, started a few, got most of the way through a couple, and failed them all. Until last week!

Last week, I successfully completed VJx, Visualizing Japan (1850s-1930s): Westernization, Protest, & Modernity through EdX.

The Tokyo Riot Graphic
The Tokyo Riot Graphic

All throughout this course, I had thoughts on what about it was so appealing to me compared to the courses I’ve taken and failed in the past.

  1. My attention span for videos (and movies and lectures and any situation where people are talking AT me) is ridiculously low – ever since high school, I’ve had to force myself to pay attention in situations like that. (Makes conferences super awesome – NOT.) So MOOCs that rely heavily on videos that are longer than 5 minutes long tend to turn me off. VJx’s videos were numerous (up to 10 per day), but nearly all clocked in at under 5 minutes. Plus, the instructors talked slowly in real time, so I was able to speed up the videos to 1.25 or 1.5 times and easily understand them.
  2. Exceptional video transcriptions let me hear AND read the video lectures at the same time. Multiple paths in my brain processed the content. (Previous MOOCs I’ve taken have had only moderately-high quality transcriptions, and the errors would disrupt the flow of learning.)
  3. We were analyzing images in this course, not texts.  I can only analyze text in printed, dead-tree form, but struggle with analyzing text on the computer.  However, I don’t have ingrained habits around image analysis, simply because I haven’t done much of it, so looking closely at images on the computer was new and novel, and allowed me to begin to build habits around image analysis.
  4. The instructors built in constant feedback and assessment into the course. Every single video had feedback questions directly below the video. Having that immediate feedback reinforced what I was learning in the videos. Plus, the types of feedback varied. There were multiple choice questions, image identification questions, image sorting exercises, and many more types of feedback and assessment. A colleague referred to this as a the “worksheet method” of learning. Who knew that I was wired to excel at that? At any rate, it worked for me.
  5. Using the discussion forum was optional, not required. One problem I have with many online communities is the noise factor: there are so many people talking at once that it can be  difficult to hear any one person. This MOOC encouraged some use of the forums, but didn’t require it, which meant I was free to dip in and out of the forums based on when I could handle the noise.
  6. The progress page was really well laid-out, and frankly, encouraged me to try to answer every single question (including on the unit quizzes and final exam) correctly. Apparently my motivation can be earned by using clear bar charts.

There is growing body of research on pedagogy in MOOCs. I don’t know if I am a typical MOOC student/learner or not. This course might not be well-suited for people who want more discussion, who learn by writing, or who are able to follow longer videos and/or lectures.  But what the folks who developed and taught VJx put together certainly hit upon my learning sweet spots.

Edited to add: Have you taken a MOOC? Did you complete it? What features about the MOOC environment worked or didn’t work for you? Which of those do you think are personal preference, and which are part of how the MOOC systems are constructed? I know I focused a fair amount in this post about how these features that worked for me might be based on personal preference – but is that really the case?