Library value & impact on learning

I've subscribed to the feeds from the Library Impact Data Project sponsored by JISC.  While I'm really interested in their results, my main interest is with their methods.  How will the researchers be able to make rational and scientific observations about library usage and academic outcomes that are valid and useful?  Well, the answer is, not easily, as this recent post explains.  The institutions in this project are all in Great Britain, which has more centralized data, such as "UCAS points" for getting into college, as well as demographic data.  In addition, the individual institutions require all students to log into the Library's resources individually, thus providing online usage for each student.  I wanted to find out what they could learn with such a relatively rich set of data.

The latest posting does suggest that even this level of data granularity is not enough.  There were problems in distinguishing "year of study" for each student (e.g. comparing usage for those in their first year versus in their second year, etc.).  This is important because usage does vary by year of study in and of itself, so it wouldn't be possible to control for this variation when examining the impact of other factors.  Their workarounds  (such as looking only at one year) effectively reduced their sample size to such an extent that statistical analysis of the results was not possible.  Finally, they had to convert one of their most key variables from a continuous (amount of time using the library) to a dichotomous variable (used/didn't use).  This, by their own admission, makes it more difficult to see the true relationship of usage to, say, completion rates.  


But given these "health warnings" of the data, they did share some intriguing results, notably that library usage is strongly and significantly associated with dropping out:

If you do not use the library, you are over seven times more likely to drop out of your degree: 7.19, to be precise. If you do not download PDFs, you are 7.89 times more likely to drop out of your degree.  Library PC usage also has a relationship with dropping out, although in this case not using the PCs makes you 2.82 times more likely to drop out of your degree.

While there may be some concerns about the levels of granularity of the measures, their strength should be indicative that there is some sort of relationship between usage and dropping out.  This, of course, should not surprise us considerably, and the results are certainly not counter-intuitive.  Furthermore, this is solely a correlation NOT an indication of causation - but I do like what is said about these results:

What it does offer, though, is a kind of ‘early warning system’. If your students aren’t using the library, it might be worth checking in with them to make sure everything is alright. Not using the library doesn’t automatically mean you’re at risk of dropping out – in fact, the number of students who don’t use the library and don’t drop out is much, much higher than the number who do leave their course prematurely. But library usage data could be another tool within the armoury of student support services, one part of a complex picture which helps them to understand which students might be at risk of failing to complete their degree.

This posting also demonstrates to me that even when with a much richer and granular data set, there will still be problems that through a monkey wrench into the best-laid plans.