Library Hat Rotating Header Image

Web

Cybersecurity, Usability, Online Privacy, and Digital Surveillance

** This post was originally published in ACRL TechConnect on May. 9, 2016.***

Cybersecurity is an interesting and important topic, one closely connected to those of online privacy and digital surveillance. Many of us know that it is difficult to keep things private on the Internet. The Internet was invented to share things with others quickly, and it excels at that job. Businesses that process transactions with customers and store the information online are responsible for keeping that information private. No one wants social security numbers, credit card information, medical history, or personal e-mails shared with the world. We expect and trust banks, online stores, and our doctor’s offices to keep our information safe and secure.

However, keeping private information safe and secure is a challenging task. We have all heard of security breaches at J.P Morgan, Target, Sony, Anthem Blue Cross and Blue Shield, the Office of Personnel Management of the U.S. federal government, University of Maryland at College Park, and Indiana University. Sometimes, a data breach takes place when an institution fails to patch a hole in its network systems. Sometimes, people fall for a phishing scam, or a virus in a user’s computer infects the target system. Other times, online companies compile customer data into personal profiles. The profiles are then sold to data brokers and on into the hands of malicious hackers and criminals.

https://www.flickr.com/photos/topgold/4978430615

Image from Flickr – https://www.flickr.com/photos/topgold/4978430615

Cybersecurity vs. Usability

To prevent such a data breach, institutional IT staff are trained to protect their systems against vulnerabilities and intrusion attempts. Employees and end users are educated to be careful about dealing with institutional or customers’ data. There are systematic measures that organizations can implement such as two-factor authentication, stringent password requirements, and locking accounts after a certain number of failed login attempts.

While these measures strengthen an institution’s defense against cyberattacks, they may negatively affect the usability of the system, lowering users’ productivity. As a simple example, security measures like a CAPTCHA can cause an accessibility issue for people with disabilities.

Or imagine that a university IT office concerned about the data security of cloud services starts requiring all faculty, students, and staff to only use cloud services that are SOC 2 Type II certified as an another example. SOC stands for “Service Organization Controls.” It consists of a series of standards that measure how well a given service organization keeps its information secure. For a business to be SOC 2 certified, it must demonstrate that it has sufficient policies and strategies that will satisfactorily protect its clients’ data in five areas known as “Trust Services Principles.” Those include the security of the service provider’s system, the processing integrity of this system, the availability of the system, the privacy of personal information that the service provider collects, retains, uses, discloses, and disposes of for its clients, and the confidentiality of the information that the service provider’s system processes or maintains for the clients. The SOC 2 Type II certification means that the business had maintained relevant security policies and procedures over a period of at least six months, and therefore it is a good indicator that the business will keep the clients’ sensitive data secure. The Dropbox for Business is SOC 2 certified, but it costs money. The free version is not as secure, but many faculty, students, and staff in academia use it frequently for collaboration. If a university IT office simply bans people from using the free version of Dropbox without offering an alternative that is as easy to use as Dropbox, people will undoubtedly suffer.

Some of you may know that the USPS website does not provide a way to reset the password for users who forgot their usernames. They are instead asked to create a new account. If they remember the account username but enter the wrong answers to the two security questions more than twice, the system also automatically locks their accounts for a certain period of time. Again, users have to create a new account. Clearly, the system that does not allow the password reset for those forgetful users is more secure than the one that does. However, in reality, this security measure creates a huge usability issue because average users do forget their passwords and the answers to the security questions that they set up themselves. It’s not hard to guess how frustrated people will be when they realize that they entered a wrong mailing address for mail forwarding and are now unable to get back into the system to correct because they cannot remember their passwords nor the answers to their security questions.

To give an example related to libraries, a library may decide to block all international traffic to their licensed e-resources to prevent foreign hackers who have gotten hold of the username and password of a legitimate user from accessing those e-resources. This would certainly help libraries to avoid a potential breach of licensing terms in advance and spare them from having to shut down compromised user accounts one by one whenever those are found. However, this would make it impossible for legitimate users traveling outside of the country to access those e-resources as well, which many users would find it unacceptable. Furthermore, malicious hackers would probably just use a proxy to make their IP address appear to be located in the U.S. anyway.

What would users do if their organization requires them to reset passwords on a weekly basis for their work computers and several or more systems that they also use constantly for work? While this may strengthen the security of those systems, it’s easy to see that it will be a nightmare having to reset all those passwords every week and keeping track of them not to forget or mix them up. Most likely, they will start using less complicated passwords or even begin to adopt just one password for all different services. Some may even stick to the same password every time the system requires them to reset it unless the system automatically detects the previous password and prevents the users from continuing to use the same one. Ill-thought-out cybersecurity measures can easily backfire.

Security is important, but users also want to be able to do their job without being bogged down by unwieldy cybersecurity measures. The more user-friendly and the simpler the cybersecurity guidelines are to follow, the more users will observe them, thereby making a network more secure. Users who face cumbersome and complicated security measures may ignore or try to bypass them, increasing security risks.

Image from Flickr - https://www.flickr.com/photos/topgold/4978430615

Image from Flickr – https://www.flickr.com/photos/topgold/4978430615

Cybersecurity vs. Privacy

Usability and productivity may be a small issue, however, compared to the risk of mass surveillance resulting from aggressive security measures. In 2013, the Guardian reported that the communication records of millions of people were being collected by the National Security Agency (NSA) in bulk, regardless of suspicion of wrongdoing. A secret court order prohibited Verizon from disclosing the NSA’s information request. After a cyberattack against the University of California at Los Angeles, the University of California system installed a device that is capable of capturing, analyzing, and storing all network traffic to and from the campus for over 30 days. This security monitoring was implemented secretly without consulting or notifying the faculty and those who would be subject to the monitoring. The San Francisco Chronicle reported the IT staff who installed the system were given strict instructions not to reveal it was taking place. Selected committee members on the campus were told to keep this information to themselves.

The invasion of privacy and the lack of transparency in these network monitoring programs has caused great controversy. Such wide and indiscriminate monitoring programs must have a very good justification and offer clear answers to vital questions such as what exactly will be collected, who will have access to the collected information, when and how the information will be used, what controls will be put in place to prevent the information from being used for unrelated purposes, and how the information will be disposed of.

We have recently seen another case in which security concerns conflicted with people’s right to privacy. In February 2016, the FBI requested Apple to create a backdoor application that will bypass the current security measure in place in its iOS. This was because the FBI wanted to unlock an iPhone 5C recovered from one of the shooters in San Bernadino shooting incident. Apple iOS secures users’ devices by permanently erasing all data when a wrong password is entered more than ten times if people choose to activate this option in the iOS setting. The FBI’s request was met with strong opposition from Apple and others. Such a backdoor application can easily be exploited for illegal purposes by black hat hackers, for unjustified privacy infringement by other capable parties, and even for dictatorship by governments. Apple refused to comply with the request, and the court hearing was to take place in March 22. The FBI, however, withdrew the request saying that it found a way to hack into the phone in question without Apple’s help. Now, Apple has to figure out what the vulnerability in their iOS if it wants its encryption mechanism to be foolproof. In the meanwhile, iOS users know that their data is no longer as secure as they once thought.

Around the same time, the Senate’s draft bill titled as “Compliance with Court Orders Act of 2016,” proposed that people should be required to comply with any authorized court order for data and that if that data is “unintelligible” – meaning encrypted – then it must be decrypted for the court. This bill is problematic because it practically nullifies the efficacy of any end-to-end encryption, which we use everyday from our iPhones to messaging services like Whatsapp and Signal.

Because security is essential to privacy, it is ironic that certain cybersecurity measures are used to greatly invade privacy rather than protect it. Because we do not always fully understand how the technology actually works or how it can be exploited for both good and bad purposes, we need to be careful about giving blank permission to any party to access, collect, and use our private data without clear understanding, oversight, and consent. As we share more and more information online, cyberattacks will only increase, and organizations and the government will struggle even more to balance privacy concerns with security issues.

Why Libraries Should Advocate for Online Privacy?

The fact that people may no longer have privacy on the Web should concern libraries. Historically, libraries have been strong advocates of intellectual freedom striving to keep patron’s data safe and protected from the unwanted eyes of the authorities. As librarians, we believe in people’s right to read, think, and speak freely and privately as long as such an act itself does not pose harm to others. The Library Freedom Project is an example that reflects this belief held strongly within the library community. It educates librarians and their local communities about surveillance threats, privacy rights and law, and privacy-protecting technology tools to help safeguard digital freedom, and helped the Kilton Public Library in Lebanon, New Hampshire, to become the first library to operate a Tor exit relay, to provide anonymity for patrons while they browse the Internet at the library.

New technologies brought us the unprecedented convenience of collecting, storing, and sharing massive amount of sensitive data online. But the fact that such sensitive data can be easily exploited by falling into the wrong hands created also the unparalleled level of potential invasion of privacy. While the majority of librarians take a very strong stance in favor of intellectual freedom and against censorship, it is often hard to discern a correct stance on online privacy particularly when it is pitted against cybersecurity. Some even argue that those who have nothing to hide do not need their privacy at all.

However, privacy is not equivalent to hiding a wrongdoing. Nor do people keep certain things secrets because those things are necessarily illegal or unethical. Being watched 24/7 will drive any person crazy whether s/he is guilty of any wrongdoing or not. Privacy allows us safe space to form our thoughts and consider our actions on our own without being subject to others’ eyes and judgments. Even in the absence of actual massive surveillance, just the belief that one can be placed under surveillance at any moment is sufficient to trigger self-censorship and negatively affects one’s thoughts, ideas, creativity, imagination, choices, and actions, making people more conformist and compliant. This is further corroborated by the recent study from Oxford University, which provides empirical evidence that the mere existence of a surveillance state breeds fear and conformity and stifles free expression. Privacy is an essential part of being human, not some trivial condition that we can do without in the face of a greater concern. That’s why many people under political dictatorship continue to choose death over life under mass surveillance and censorship in their fight for freedom and privacy.

The Electronic Frontier Foundation states that privacy means respect for individuals’ autonomy, anonymous speech, and the right to free association. We want to live as autonomous human beings free to speak our minds and think on our own. If part of a library’s mission is to contribute to helping people to become such autonomous human beings through learning and sharing knowledge with one another without having to worry about being observed and/or censored, libraries should advocate for people’s privacy both online and offline as well as in all forms of communication technologies and devices.

Using the Stripe API to Collect Library Fines by Accepting Online Payments

*** This post was originally published in ACRL TechConnect on Sep. 10, 2014.***

Recently, my library has been considering accepting library fines via online. Currently, many library fines of a small amount that many people owe are hard to collect. As a sum, the amount is significant enough. But each individual fines often do not warrant even the cost for the postage and the staff work that goes into creating and sending out the fine notice letter. Libraries that are able to collect fines through the bursar’s office of their parent institutions may have a better chance at collecting those fines. However, others can only expect patrons to show up with or to mail a check to clear their fines. Offering an online payment option for library fines is one way to make the library service more user-friendly to those patrons who are too busy to visit the library in person or to mail a check but are willing to pay online with their credit cards.

If you are new to the world of online payment, there are several terms you need to become familiar with. The following information from the article in SixRevisions is very useful to understand those terms.1

  • ACH (Automated Clearing House) payments: Electronic credit and debit transfers. Most payment solutions use ACH to send money (minus fees) to their customers.
  • Merchant Account: A bank account that allows a customer to receive payments through credit or debit cards. Merchant providers are required to obey regulations established by card associations. Many processors act as both the merchant account as well as the payment gateway.
  • Payment Gateway: The middleman between the merchant and their sponsoring bank. It allows merchants to securely pass credit card information between the customer and the merchant and also between merchant and the payment processor.
  • Payment Processor: A company that a merchant uses to handle credit card transactions. Payment processors implement anti-fraud measures to ensure that both the front-facing customer and the merchant are protected.
  • PCI (the Payment Card Industry) Compliance: A merchant or payment gateway must set up their payment environment in a way that meets the Payment Card Industry Data Security Standard (PCI DSS).

Often, the same company functions as both payment gateway and payment processor, thereby processing the credit card payment securely. Such a product is called ‘Online payment system.’ Meyer’s article I have cited above also lists 10 popular online payment systems: Stripe, Authorize.Net, PayPal, Google Checkout, Amazon Payments, Dwolla, Braintree, Samurai by FeeFighters, WePay, and 2Checkout. Bear in mind that different payment gateways, merchant accounts, and bank accounts may or may not work together, your bank may or may not work as a merchant account, and your library may or may not have a merchant account. 2

Also note that there are fees in using online payment systems like these and that different systems have different pay structures. For example, Authorize.net has the $99 setup fee and then charges $20 per month plus a $0.10 per-transaction fee. Stripe charges 2.9% + $0.30 per transaction with no setup or monthly fees. Fees for mobile payment solutions with a physical card reader such as Square may go up much higher.

Among various online payment systems, I picked Stripe because it was recommended on the Code4Lib listserv. One of the advantages for using Stripe is that it acts as both the payment gateway and the merchant account. What this means is that your library does not have to have a merchant account to accept payment online. Another big advantage of using Stripe is that you do not have to worry about the PCI compliance part of your website because the Stripe API uses a clever way to send the sensitive credit card information over to the Stripe server while keeping your local server, on which your payment form sits, completely blind to such sensitive data. I will explain this in more detail later in this post.

Below I will share some of the code that I have used to set up Stripe as my library’s online payment option for testing. This may be of interest to you if you are thinking about offering online payment as an option for your patrons or if you are simply interested in how an online payment API works. Even if your library doesn’t need to collect library fines via online, an online payment option can be a handy tool for a small-scale fund-raising drive or donation.

The first step to take to make Stripe work is getting an API keys. You do not have to create an account to get API keys for testing. But if you are going to work on your code more than one day, it’s probably worth getting an account. Stripe API has excellent documentation. I have read ‘Getting Started’ section and then jumped over to the ‘Examples’ section, which can quickly get you off the ground. (https://stripe.com/docs/examples) I found an example by Daniel Schröter in GitHub from the list of examples in the Stripe’s Examples section and decided to test out. (https://github.com/myg0v/Simple-Bootstrap-Stripe-Payment-Form) Most of the time, getting an example code requires some probing and tweaking such as getting all the required library downloaded and sorting out the paths in the code and adding API keys. This one required relatively little work.

Now, let’s take a look at the form that this code creates.

borrowedcode

In order to create a form of my own for testing, I decided to change a few things in the code.

  1. Add Patron & Payment Details.
  2. Allow custom amount for payment.
  3. Change the currency from Euro to US dollars.
  4. Configure the validation for new fields.
  5. Hide the payment form once the charge goes through instead of showing the payment form below the payment success message.

html

4. can be done as follows. The client-side validation is performed by Bootstrapvalidator jQuery Plugin. So you need to get the syntax correct to get the code, which now has new fields, to work properly.
validator

This is the Javascript that allows you to send the data submitted to your payment form to the Stripe server. First, include the Stripe JS library (line 24). Include JQuery, Bootstrap, Bootstrap Form Helpers plugin, and Bootstrap Validator plugin (line 25-28). The next block of code includes an event handler for the form, which send the payment information to the Stripe via AJAX when the form is submitted. Stripe will validate the payment information and then return a token that identifies this particular transaction.

jspart

When the token is received, this code calls for the function, stripeResponseHandler(). This function, stripeResponseHandler() checks if the Stripe server did not return any error upon receiving the payment information and, if no error has been returned, attaches the token information to the form and submits the form.

jspart2

The server-side PHP script then checks if the Stripe token has been received and, if so, creates a charge to send it to Stripe as shown below. I am using PHP here, but Stripe API supports many other languages than PHP such as Ruby and Python. So you have many options. The real payment amount appears here as part of the charge array in line 326. If the charge succeeds, the payment success message is stored in a div to be displayed.

phppart

The reason why you do not have to worry about the PCI compliance with Stripe is that Stripe API asks to receive the payment information via AJAX and the input fields of sensitive information does not have the name attribute and value. (See below for the Card Holder Name and Card Number information as an example; Click to bring up the clear version of the image.)  By omitting the name attribute and value, the local server where the online form sits is deprived of any means to retrieve the information in those input fields submitted through the form. Since sensitive information does not touch the local server at all, PCI compliance for the local server becomes no concern. To clarify, not all fields in the payment form need to be deprived of the name attribute. Only the sensitive fields that you do not want your web server to have access to need to be protected this way. Here, for example, I am assigning the name attribute and value to fields such as name and e-mail in order to use them later to send a e-mail receipt.

(NB. Please click images to see the enlarged version.)

Screen Shot 2014-08-17 at 8.01.08 PM

Now, the modified form has ‘Fee Category’, custom ‘Payment Amount,’ and some other information relevant to the billing purpose of my library.

updated

When the payment succeeds, the page changes to display the following message.

success

Stripe provides a number of fake card numbers for testing. So you can test various cases of failures. The Stripe website also displays all payments and related tokens and charges that are associated with those payments. This greatly helps troubleshooting. One thing that I noticed while troubleshooting is that Stripe logs sometimes do lag behind. That is, when a payment would succeed, associated token and charge may not appear under the “Logs” section immediately. But you will see the payment shows up in the log. So you will know that associated token and charge will eventually appear in the log later.

recent_payment

Once you are ready to test real payment transactions, you need to flip the switch from TEST to LIVE located on the top left corner. You will also need to replace your API keys for ‘TESTING’ (both secret and public) with those for ‘LIVE’ transaction. One more thing that is needed before making your library getting paid with real money online is setting up SSL (Secure Sockets Layer) for your live online payment page. This is not required for testing but necessary for processing live payment transactions. It is not a very complicated work. So don’t be discouraged at this point. You just have to buy a security certificate and put it in your Web server. Speak to your system administrator for how to get the SSL set up for your payment page. More information about setting up SSL can be found in the Stripe documentation I just linked above.

My library has not yet gone live with this online payment option. Before we do, I may make some more modifications to the code to fit the staff workflow better, which is still being mapped out. I am also planning to place the online payment page behind the university’s Shibboleth authentication in order to cut down spam and save some tedious data entry by library patrons by getting their information such as name, university email, student/faculty/staff ID number directly from the campus directory exposed through Shibboleth and automatically inserting it into the payment form fields.

In this post, I have described my experience of testing out the Stripe API as an online payment solution. As I have mentioned above, however, there are many other online payment systems out there. Depending your library’s environment and financial setup, different solutions may work better than others. To me, not having to worry about the PCI compliance by using Stripe was a big plus. If your library accepts online payment, please share what solution you chose and what factors led you to the particular online payment system in the comments.

* This post has been based upon my recent presentation, “Accepting Online Payment for Your Library and ‘Stripe’ as an Example”, given at the Code4Lib DC Unconference. See the slides  below..

Notes 
  1. Meyer, Rosston. “10 Excellent Online Payment Systems.” Six Revisions, May 15, 2012. http://sixrevisions.com/tools/online-payment-systems/.
  2. Ullman, Larry. “Introduction to Stripe.” Larry Ullman, October 10, 2012. http://www.larryullman.com/2012/10/10/introduction-to-stripe/.

Query a Google Spreadsheet like a Database with Google Visualization API Query Language

***  This post was originally published in ACRL TechConnect on Dec. 4, 2013. ***

Libraries make much use of spreadsheets. Spreadsheets are easy to create, and most library staff are familiar with how to use them. But they can quickly get unwieldy as more and more data are entered. The more rows and columns a spreadsheet has, the more difficult it is to browse and quickly identify specific information. Creating a searchable web application with a database at the back-end is a good solution since it will let users to quickly perform a custom search and filter out unnecessary information. But due to the staff time and expertise it requires, creating a full-fledged searchable web database application is not always a feasible option at many libraries.

Creating a MS Access custom database or using a free service such as Zoho can be an alternative to creating a searchable web database application. But providing a read-only view for MS Access database can be tricky although possible. MS Access is also software locally installed in each PC and therefore not necessarily available for the library staff when they are not with their work PCs on which MS Access is installed. Zoho Creator offers a way to easily convert a spreadsheet into a database, but its free version service has very limited features such as maximum 3 users, 1,000 records, and 200 MB storage.

Google Visualization API Query Language provides a quick and easy way to query a Google spreadsheet and return and display a selective set of data without actually converting a spreadsheet into a database. You can display the query result in the form of a HTML table, which can be served as a stand-alone webpage. All you have to do is to construct a custom URL.

A free version of Google spreadsheet has a limit in size and complexity. For example, one free Google spreadsheet can have no more than 400, 000 total cells. But you can purchase more Google Drive storage and also query multiple Google spreadsheets (or even your own custom databases) by using Google Visualization API Query Language and Google Chart Libraries together. (This will be the topic of my next post. You can also see the examples of using Google Chart Libraries and Google Visualization API Query Language together in my presentation slides at the end of this post.)

In this post, I will explain the parameters of Google Visualization API Query Language and how to construct a custom URL that will query, return, and display a selective set of data in the form of an HTML page.

A. Display a Google Spreadsheet as an HTML page

The first step is to identify the URL of the Google spreadsheet of your choice.

The URL below opens up the third sheet (Sheet 3) of a specific Google spreadsheet. There are two parameters you need to pay attention inside the URL: key and gid.

https://docs.google.com/spreadsheet/ccc?key=0AqAPbBT_k2VUdDc3aC1xS2o0c2ZmaVpOQWkyY0l1eVE&usp=drive_web#gid=2

This breaks down the parameters in a way that is easier to view:

  • https://docs.google.com/spreadsheet/ccc
    ?key=0AqAPbBT_k2VUdDc3aC1xS2o0c2ZmaVpOQWkyY0l1eVE
    &usp=drive_web

    #gid=2

Key is a unique identifier to each Google spreadsheet. So you need to use that to cretee a custom URL later that will query and display the data in this spreadsheet. Gid specifies which sheet in the spreadsheet you are opening up. The gid for the first sheet is 0; the gid for the third sheet is 2.

Screen Shot 2013-11-27 at 9.44.29 AM

Let’s first see how Google Visualization API returns the spreadsheet data as a DataTable object. This is only for those who are curious about what goes on behind the scenes. You can see that for this view, the URL is slightly different but the values of the key and the gid parameter stay the same.

https://spreadsheets.google.com/tq?&tq=&key=0AqAPbBT_k2VUdDc3aC1xS2o0c2ZmaVpOQWkyY0l1eVE&gid=2

Screen Shot 2013-11-27 at 9.56.03 AM

In order to display the same result as an independent HTML page, all you need to do is to take the key and the gid parameter values of your own Google spreadsheet and construct the custom URL following the same pattern shown below.

  • https://spreadsheets.google.com
    /tq?tqx=out:html&tq=
    &key=0AqAPbBT_k2VUdDc3aC1xS2o0c2ZmaVpOQWkyY0l1eVE
    &gid=2

https://spreadsheets.google.com/tq?tqx=out:html&tq=&key=0AqAPbBT_k2VUdDc3aC1xS2o0c2ZmaVpOQWkyY0l1eVE&gid=2

Screen Shot 2013-11-27 at 9.59.11 AM

By the way, if the URL you created doesn’t work, it is probably because you have not encoded it properly. Try this handy URL encoder/decoder page to encode it by hand or you can use JavaScript encodeURIComponent() function.
Also if you want the URL to display the query result without people logging into Google Drive first, make sure to set the permission setting of the spreadsheet to be public. On the other hand, if you need to control access to the spreadsheet only to a number of users, you have to remind your users to first go to Google Drive webpage and log in with their Google account before clicking your URLs. Only when the users are logged into Google Drive, they will be able see the query result.

B. How to Query a Google Spreadsheet

We have seen how to create a URL to show an entire sheet of a Google spreadsheet as an HTML page above. Now let’s do some querying, so that we can pick and choose what data the table is going to display instead of the whole sheet. That’s where the Query Language comes in handy.

Here is an example spreadsheet with over 50 columns and 500 rows.

  • https://docs.google.com/spreadsheet/ccc?
    key=0AqAPbBT_k2VUdDFYamtHdkFqVHZ4VXZXSVVraGxacEE
    &usp=drive_web
    #gid=0

https://docs.google.com/spreadsheet/ccc?key=0AqAPbBT_k2VUdDFYamtHdkFqVHZ4VXZXSVVraGxacEE&usp=drive_web#gid=0

Screen Shot 2013-11-27 at 10.15.41 AM

What I want to do is to show only column B, C, D, F where C contains ‘Florida.’ How do I do this? Remember the URL we created to show the entire sheet above?

  • https://spreadsheets.google.com/tq?tqx=out:html&tq=&key=___&gid=___

There we had no value for the tq parameter. This is where we insert our query.

Google Visualization API Query Language is pretty much the same as SQL. So if you are familiar with SQL, forming a query is dead simple. If you aren’t SQL is also easy to learn.

  • The query should be written like this:
    SELECT B, C, D, F WHERE C CONTAINS ‘Florida’
  • After encoding it properly, you get something like this:
    SELECT%20B%2C%20C%2C%20D%2C%20F%20WHERE%20C%20CONTAINS%20%27Florida%27
  • Add it to the tq parameter and don’t forget to also specify the key:
    https://spreadsheets.google.com/tq?tqx=out:html&tq=SELECT%20B%2C%20C%2C%20D%2C%20F%20WHERE%20C%20CONTAINS%20%27Florida%27
    &key=0AqAPbBT_k2VUdEtXYXdLdjM0TXY1YUVhMk9jeUQ0NkE

I am omitting the gid parameter here because there is only one sheet in this spreadsheet but you can add it if you would like. You can also omit it if the sheet you want is the first sheet. Ta-da!

Screen Shot 2013-11-27 at 10.26.13 AM

Compare this with the original spreadsheet view. I am sure you can appreciate how the small effort put into creating a URL pays back in terms of viewing an unwieldy large spreadsheet manageable.

You can also easily incorporate functions such as count() or sum() into your query to get an overview of the data you have in the spreadsheet.

  • select D,F count(C) where (B contains ‘author name’) group by D, F

For example, this query above shows how many articles a specific author published per year in each journal. The screenshot of the result is below and you can see it for yourself here: https://spreadsheets.google.com/tq?tqx=out:html&tq=select+D,F,count(C)+where+%28B+contains+%27Agoulnik%27%29+group+by+D,F&key=0AqAPbBT_k2VUdEtXYXdLdjM0TXY1YUVhMk9jeUQ0NkE

Screen Shot 2013-11-27 at 11.34.25 AM

Take this spread sheet as another example.

libbudgetfake

This simple query below displays the library budget by year. For those who are unfamiliar with ‘pivot‘, pivot table is a data summarization tool. The query below asks the spreadsheet to calculate the total of all the values in the B column (Budget amount for each category) by the values found in the C column (Years).

Screen Shot 2013-11-27 at 11.46.49 AM

This is another example of querying the spreadsheet connected to my library’s Literature Search request form. The following query asks the spreadsheet to count the number of literature search requests by Research Topic (=column I) that were received in 2011 (=column G) grouped by the values in the column C, i.e. College of Medicine Faculty or College of Medicine Staff.

  • select C, count(I) where (G contains ‘2011’) group by C

litsearch

C. More Querying Options

There are many more things you can do with a custom query. Google has an extensive documentation that is easy to follow: https://developers.google.com/chart/interactive/docs/querylanguage#Language_Syntax

These are just a few examples.

  • ORDER BY __ DESC
    : Order the results in the descending order of the column of your choice. Without ‘DESC,’ the result will be listed in the ascending order.
  • LIMIT 5
    : Limit the number of results. Combined with ‘Order by’ you can quickly filter the results by the most recent or the oldest items.

My presentation slides given at the 2013 LITA Forum below includes more detailed information about Google Visualization API Query Language, parameters, and other options as well as how to use Google Chart Libraries in combination with Google Visualization API Query Language for data visualization, which is the topic of my next post.

Happy querying Google Spreadsheet!

 

Redesigning the Item Record Summary View in a Library Catalog and a Discovery Interface

***  This post was originally published in ACRL TechConnect on Oct. 15, 2013. ***

A. Oh, the Library Catalog

Almost all librarians have a love-hate relationship with their library catalogs (OPAC), which are used by library patrons. Interestingly enough, I hear a lot more complaints about the library catalog from librarians than patrons. Sometimes it is about the catalog missing certain information that should be there for patrons. But many other times, it’s about how crowded the search results display looks. We actually all want a clean-looking, easy-to-navigate, and efficient-to-use library catalog. But of course, it is much easier to complain than to come up with an viable alternative.

Aaron Schmidt has recently put forth an alternative design for a library item record. In his blog post, he suggests a library catalog shifts its focus from the bibliographic information (or metadata if not a book) of a library item to a patron’s tasks performed in relation to the library item so that the catalog functions more as “a tool that prioritizes helping people accomplish their tasks, whereby bibliographic data exists quietly in the background and is exposed only when useful.” This is a great point. Throwing all the information at once to a user only overwhelms her/him. Schmidt’s sketch provides a good starting point to rethink how to design the library catalog’s search results display.

Screen Shot 2013-10-09 at 1.34.08 PM

From the blog post, “Catalog Design” by Aaron Schmidt

B. Thinking about Alternative Display Design

The example above is, of course, too simple to apply to the library catalog of an academic library straight away. For an usual academic library patron to determine whether s/he wants to either check out or reserve the item, s/he is likely to need a little more information than the book title, the author, and the book image. For example, students who look for textbooks, the edition information as well as the year of publication are important. But I take it that Schmidt’s point was to encourage more librarians to think about alternative designs for the library catalog rather than simply compare what is available and pick what seems to be the best among those.

Screen Shot 2013-10-09 at 1.44.36 PM

Florida International University Library Catalog – Discovery layer, Mango, provided by Florida Virtual Campus

Granted that there may be limitations in how much we can customize the search results display of a library catalog. But that is not a reason to stop thinking about what the optimal display design would be for the library catalog search results. Sketching alternatives can be in itself a good exercise in evaluating the usability of an information system even if not all of your design can be implemented.

Furthermore, more and more libraries are implementing a discovery layer over their library catalogs, which provides much more room to customize the display of search results than the traditional library catalog. Open source discovery systems such as Blacklight or VuFind provides great flexibility in customizing the search results display. Even proprietary discovery products such as Primo, EDS, Summon offer a level of customization by the libraries.

Below, I will discuss some principles to follow in sketching alternative designs for search results in a library catalog, present some of my own sketches, and show other examples implemented by other libraries or websites.

C. Principles

So, if we want to improve the item record summary display to be more user-friendly, where can we start and what kind of principles should we follow? These are the principles that I followed in coming up with my own design:

  • De-clutter.
  • Reveal just enough information that is essential to determine the next action.
  • Highlight the next action.
  • Shorten texts.

These are not new principles. They are widely discussed and followed by many web designers including librarians who participate in their libraries’ website re-design. But we rarely apply these to the library catalog because we think that the catalog is somehow beyond our control. This is not necessarily the case, however. Many libraries implement discovery layers to give a completely different and improved look from that of their ILS-es’ default display.

Creating a satisfactory design on one’s own instead of simply pointing out what doesn’t work or look good in existing designs is surprisingly hard but also a refreshing challenge. It also brings about the positive shift of focus in thinking about a library catalog from “What is the problem in the catalog?” to “What is a problem and what can we change to solve the problem?”

Below I will show my own sketches for an item record summary view for the library catalog search results. These are clearly a combination of many other designs that I found inspiring in other library catalogs. (I will provide the source of those elements later in this post.) I tried to mix and revise them so that the result would follow those four principles above as closely as possible. Check them out and also try creating your own sketches. (I used Photoshop for creating my sketches.)

D. My Own Sketches

Here is the basic book record summary view. What I tried to do here is giving just enough information for the next action but not more than that: title, author, type, year, publisher, number of library copies and holds. The next action for a patron is to check the item out. On the other hand, undecided patrons will click the title to see the detailed item record or have the detailed item record to be texted, printed, e-mailed, or to be used in other ways.

(1) A book item record

Screen Shot 2013-10-09 at 12.46.38 PM

This is a record of a book that has an available copy to check out. Only when a patron decides to check out the item, the next set of information relevant to that action – the item location and the call number – is shown.

(2) With the check-out button clicked

check out box open

If no copy is available for check-out, the best way to display the item is to signal that check-out is not possible and to highlight an alternative action. You can either do this by graying out the check-out button or by hiding the button itself.

Many assume that adding more information would automatically increase the usability of a website. While there are cases in which this could be true, often a better option is to reveal information only when it is relevant.

I decided to gray out the check-out button when there is no available copy and display the reserve button, so that patrons can place a hold. Information about how many copies the library has and how many holds are placed (“1 hold / 1 copy”) would help a patron to decide if they want to reserve the book or not.

(3) A book item record when check-out is not available

Screen Shot 2013-10-09 at 12.34.54 PM

I also sketched two other records: one for an e-Book without the cover image and the other with the cover image. Since the appropriate action in this case is reading online, a different button is shown. You may place the ‘Requires Login’ text or simply omit it because most patrons will understand that they will have to log in to read a library e-book and also the read-online button will itself prompt log in once clicked anyway.

(4) An e-book item record without a book cover

Screen Shot 2013-10-09 at 12.35.54 PM

(5) An e-book item record with a book cover

Screen Shot 2013-10-09 at 12.48.33 PM

(6) When the ‘Read Online’ button is clicked, an e-book item record with multiple links/providers

When there are multiple options for one electronic resource, those options can be presented in a similar way in which multiple copies of a physical book are shown.

Screen Shot 2013-10-09 at 12.35.22 PM

(6) A downloadable e-book item record

For a downloadable resource, changing the name of the button to ‘download’ is much more informative.

Screen Shot 2013-10-09 at 12.35.13 PM

(7) An e-journal item record

Screen Shot 2013-10-09 at 12.47.31 PM

(7) When the ‘Read Online’ button is clicked, an e-journal item record with multiple links/providers

Screen Shot 2013-10-09 at 12.41.56 PM

E. Inspirations

Needless to say, I did not come up with my sketches from scratch. Here are the library catalogs whose item record summary view inspired me.

torontopublic

Toronto Public Library catalog has an excellent item record summary view, which I used as a base for my own sketches. It provides just enough information for the summary view. The title is hyperlinked to the detailed item record, and the summary view displays the material type and the year in bod for emphasis. The big green button also clearly shows the next action to take. It also does away with unnecessary labels that are common in library catalog such as ‘Author:’ ‘Published:’ ‘Location:’ ‘Link:.’

User Experience Designer Ryan Feely, who worked on Toronto Public Library’s catalog search interface, pointed out the difference between a link and an action in his 2009 presentation “Toronto Public Library Website User Experience Results and Recommendations.” Actions need to be highlighted as a button or in some similar design to stand out to users (slide 65). And ideally, only the actions available for a given item should be displayed.

Another good point which Feely makes (slide 24) is that an icon is often the center of attention and so a different icon should be used to signify different type of materials such as a DVD or an e-Journal. Below are the icons that Toronto Public Library uses for various types of library materials that do not have unique item images. These are much more informative than the common “No image available” icon.

eAudiobooke-journal eMusic

vinyl VHS eVideo

University of Toronto Libraries has recently redesigned their library catalog to be completely responsive. Their item record summary view in the catalog is brief and clear. Each record in the summary view also uses a red and a green icon that helps patrons to determine the availability of an item quickly. The icons for citing, printing, e-mailing, or texting the item record that often show up in the catalog are hidden in the options icon at the bottom right corner. When the mouse hovers over, a variety of choices appear.

Screen Shot 2013-10-09 at 4.45.33 PM

univtoronto

Richland Library’s catalog displays library items in a grid as a default, which makes the catalog more closely resemble an online bookstore or shopping website. Patrons can also change the view to have more details shown with or without the item image. The item record summary view in the default grid view is brief and to the point. The main type of patron action, such as Hold or Download, is clearly differentiated from other links as an orange button.

richland

Screen Shot 2013-10-13 at 8.33.46 PM

Standford University Library offers a grid view (although not as the default like Richland Library). The grid view is very succinct with the item title, call number, availability information in the form of a green checkmark, and the item location.

Screen Shot 2013-10-13 at 8.37.37 PM

What is interesting about Stanford University Library catalog (using Blacklight) is that when a patron hovers its mouse over an item in the grid view, the item image displays the preview link. And when clicked, a more detailed information is shown as an overlay.

Screen Shot 2013-10-13 at 8.37.58 PM

Brigham Young University completely customized the user interface of the Primo product from ExLibris.

byu

And University of Michigan Library customized the search result display of the Summon product from SerialsSolutions.

Screen Shot 2013-10-14 at 11.16.49 PM

Here are some other item record summary views that are also fairly straightforward and uncluttered but can be improved further.

Sacramento Public Library uses the open source discovery system, VuFind, with little customization.

dcpl

I have not done an extensive survey of library catalogs to see which one has the best item record summary view. But it seemed to me that in general academic libraries are more likely to provide more information than necessary in the item record summary view and also to require patrons to click a link instead of displaying relevant information right away. For example, the ‘Check availability’ link that is shown in many library catalogs is better when it is replaced by the actual availability status of ‘available’ or ‘checked out.’ Similarly, the ‘Full-text online’ or ‘Available online’ link may be clearer with an button titled ‘Read online’ or ‘Access online.’

F. Challenges and Strategies

The biggest challenge in designing the item record summary view is to strike the balance between too little information and too much information about the item. Too little information will require patrons to review the detailed item record just to identify if the item is the one they are looking for or not.

Since librarians know many features of the library catalog, they tend to err on the side of throwing all available features into the item record summary view. But too much information not only overwhelms patrons and but also makes it hard for them to locate the most relevant information at that stage and to identify the next available action. Any information irrelevant to a given task is no more than noise to a patron.

This is not a problem unique to a library catalog but generally applicable to any system that displays search results. In their book, Designing the Search Experience , Tony Russell-Rose and Tyler Tate describes this as achieving ‘the optimal level of detail.’ (p.130)

Useful strategies for achieving the optimal level of detail for the item summary view in the case of the library catalog include:

  • Removing all unnecessary labels
  • Using appropriate visual cues to make the record less text-heavy
  • Highlighting next logical action(s) and information relevant to that action
  • Systematically guiding a patron to the actions that are relevant to a given item and her/his task in hand

Large online shopping websites, Amazon, Barnes & Noble, and eBay all make a good use of these strategies. There are no labels such as ‘price,’ ‘shipping,’ ‘review,’ etc. Amazon highlights the price and the user reviews most since those are the two most deciding factors for consumers in their browsing stage. Amazon only offers enough information for a shopper to determine if s/he is further interested in purchasing the item. So there is not even the Buy button in the summary view. Once a shopper clicks the item title link and views the detailed item record, then the buying options and the ‘Add to Cart’ button are displayed prominently.

Screen Shot 2013-10-09 at 1.21.15 PM

Barnes & Noble’s default display for search results is the grid view, and the item record summary view offers only the most essential information – the item title, material type, price, and the user ratings.

Screen Shot 2013-10-09 at 1.24.05 PM

eBay’s item record summary view also offers only the most essential information, the highest bid and the time left, while people are browsing the site deciding whether to check out the item in further detail or not.

Screen Shot 2013-10-09 at 1.28.47 PM

G. More Things to Consider

An item record summary view, which we have discussed so far, is surely the main part of the search results page. But it is only a small part of the search results display and even a smaller part of the library catalog. Optimizing the search results page, for example, entails not just re-designing the item record summary view but choosing and designing many other elements of the page such as organizing the filtering options on the left and deciding on the default and optional views. Determining the content and the display of the detailed item record is another big part of creating a user-friendly library catalog. If you are interested in this topic, Tony Russell-Rose and Tyler Tate’s book Designing the Search Experience (2013) provides an excellent overview.

Librarians are professionals trained in many uses of a searchable database, a known item search, exploring and browsing, a search with incomplete details, compiling a set of search results, locating a certain type of items only by location, type, subject, etc. But since our work is also on the operation side of a library, we often make the mistake of regarding the library catalog as one huge inventory system that should hold and display all the acquisition, cataloging, and holdings information data of the library collection. But library patrons are rarely interested in seeing such data. They are interested in identifying relevant library items and using them. All the other information is simply a guide to achieving this ultimate goal, and the library catalog is another tool in their many toolboxes.

Online shopping sites optimize their catalog to make purchase as efficient and simple as possible. Both libraries and online shopping sites share the common interests of guiding the users to one ultimate task – identifying an appropriate item for the final borrowing or access/purchase. For creating user-oriented library catalog sketches, it is helpful to check out how non-library websites are displaying their search results as well.

Screen Shot 2013-10-13 at 9.52.27 PM

music

themes

Once you start looking other examples, you will realize that there are very many ways to display search results and you will soon want to sketch your own alternative design for the search results display in the library catalog and the discovery system. What do you think would be the level of optimum detail for library items in the library catalog or the discovery interface?

Further Reading

 

 

Playing with JavaScript and JQuery – the Ebook link HTML string generator and the EZproxy bookmarklet generator

***This post has been published in ACRL TechConnect on April 8, 2013. ***

In this post, I will describe two cases in which I solved a practical problem with a little bit of JavaScript and JQuery. Check them out first here before reading the rest of the post which will explain how the code works.

  1. Library ebook link HTML string generator
  2. EZproxy bookmarklet generator – Longer version (with EZproxy Suffix)
  3. EZproxy bookmarklet generator – Shorter version (with EZproxy Prefix)

Source: http://www.flickr.com/photos/albaum/448573998/

1. Library ebook link HTML string generator

If you are managing a library website, you will be using a lot of hyperlinks for library resources. Sometimes you can distribute some of the repetitive tasks to student workers, but they are usually not familiar with HTML. I had a situation in which I needed to add a number of links for library e-books for my library’s Course E-book LibGuide page, when I was swamped with many other projects at the same time. So I wondered if I could somehow still use the library’s student assistant’s help by creating an providing a simple tool, so that the student only needs to input the link title and url and get me the result as HTML. This way, I can still delegate some work when I am swamped with other projects that require my attention. (N.B. Needless to say, this doesn’t mean that what I did was the best way to use the student assistance for this type of work. I didn’t want them to edit the page directly because this page had tabs tabs and the student using the WYSWYG editor might inadvertently remove part of the tabbed box code.)

The following code exactly does that.

This HTML form takes an e-book title and the link to the book as input and spits out a hyperlink as a list item as a result. For example, if you fill in the title field with ‘Bradley’s Neurology in Clinical Practice’ and the link field with its url: http://ezproxy.fiu.edu/login?url=http://www.mdconsult.com/public/book/view?title=Daroff:+Bradley’s+Neurology+in+Clinical+Practice, then the result would be shown in the text area : <li><a href=”http://ezproxy.fiu.edu/login?url=http://www.mdconsult.com/public/book/view?title=Daroff:+Bradley’s+Neurology+in+Clinical+Practice”> Bradley’s Neurology in Clinical Practice</a></li>  I also wanted the library student assistant to be able to do this for many e-books at once and just send me the whole set of HTML snippets that cover all ebooks. So after running the form once, if one fills out the title and the link field with another e-book information, the result would be added to the end of the previous HTML string and be displayed in the text area. The result would be like this: <li><a href=”http://ezproxy.fiu.edu/login?url=http://www.mdconsult.com/public/book/view?title=Daroff:+Bradley’s+Neurology+in+Clinical+Practice”> Bradley’s Neurology in Clinical Practice</a></li><li><a href=”http://ezproxy.fiu.edu/login?url=http://www.accessmedicine.com/resourceTOC.aspx?resourceID=64″>Cardiovascular Physiology</a></li>. Since the code is in the text area, the student can also edit if there was any error when s/he was filling out the form after clicking the button ‘Send to Text Area’.

Now, let’s take a look at what is going on behind the scene. This is the entire html file. The Javascript/JQuery code that is generating the html string in the text area is from line 22-32.

<html>
<head>
<script type="text/javascript" src="http://ajax.googleapis.com/ajax/libs/jquery/1.8.0/jquery.min.js"></script>
</head>
<body>
	<p>
		This page creates the html-friendly string for a link with a title and a url. 
		<ul>
		<li>Fill out the form below and click the button. </li>
		<li>If the link is messy, clean up first by using the <a href="http://meyerweb.com/eric/tools/dencoder/">URL decoder</a>.</li>
		<li>Copy and paste the result inside the text area after you are done.</li>
		</ul>	
	</p>
	<p>
		Title: <input type="text" id="title1" size="100"/><br/>
		Link: <input type="text" id="link1" size="100"/><br/>
		<button id="b1">Send to Text Area</button>
	</p>
	<p id="result1"></p>
	<textarea rows="20" cols="80"></textarea>

	<script type="text/javascript">
	$(document).ready(function(){	
	  $("#b1").click(function(){
	  //  alert('<a href="'+$("#link").val()+'">' + $("#title").val()+"</a>");
	    $('#result1').text('<li><a href="'+$("#link1").val()+'">' + $("#title1").val()+"</a></li>");
	    var res1='<li><a href="'+$("#link1").val()+'">' + $("#title1").val()+"</a></li>";
	    $('textarea').val($('textarea').val()+res1);
	 //   $('textarea').text(res1);
	  });
	}); // doc ready
	</script>

</body>
</html>

Since I am using JQuery I am starting with the obligatory $(document).ready in line 2. In line 3,  I am giving a callback function that will be executed when the #b1 – the button with the id of b1 in line 17 above- is clicked.  Line 4 is commented out. I used this initially to test if I am getting the right string out of the input from the title and the link field using the JS alert. Line 5 is filling the p tag with the id of result 1 in line 19 above with the thus-created string. The string is also saved in variable res1 in line 7. Then it is attached to the content of the textarea field in line 8. Line 9 is commented out. If you use line 9 instead of line 8, any existing content in the textarea will be removed and only the new string created from the title and the link field will show up in the text area.

<script type="text/javascript">
	$(document).ready(function(){	
	  $("#b1").click(function(){
	  //  alert('<a href="'+$("#link").val()+'">' + $("#title").val()+"</a>");
	    $('#result1').text('<li><a href="'+$("#link1").val()+'">' + $("#title1").val()+"</a></li>");
	    var res1='<li><a href="'+$("#link1").val()+'">' + $("#title1").val()+"</a></li>";
	    $('textarea').val($('textarea').val()+res1);
	 //   $('textarea').text(res1);
	  });
	}); // doc ready
</script>

You do not have to know a lot about scripting to solve a simple task like this!

2. EZproxy bookmarklet generator – Longer version

My second example is a bookmarklet that reloads the current page through a specific EZproxy system.

If you think that this bookmarklet reinvents the wheel since the LibX toolbar already does this, you are correct. And also if you are a librarian working with e-resources, you already know to add the EZproxy suffix at the end of the domain name of the url when a patron asks if a certain article on a web page is available through the library or not. But I found that no matter how many times I explain this trick of adding the EZproxy suffix to patrons, the trick doesn’t seem to stick in their busy minds. Also, many doctors and medical students, who are the primary patrons of my library, work at the computers in hospitals and they do not have the necessary privilege to install a toolbar on those computers. But they can create a bookmark.

Similarly, many students asked me why there is no LibX toolbar for their mobile devices unlike in their school laptops. (In the medical school where I work, all students are required to purchase a school-designated laptop; this laptop is pre-configured with all the necessary programs including the library LibX toolbar.) Well, mobile devices are not exactly computers and so the browser toolbar doesn’t work. But students want an alternative and they can create a bookmark on their tablets and smartphones. So the proxy bookmarklet is still a worthwhile tool for the mobile device users.

This is where the bookmarklet is: http://htmlpreview.github.com/?https://github.com/bohyunkim/examples/blob/master/bkmklt.html. To test, drag the link on the top that says Full-Text from FIU Library to your bookmark toolbar. Then go to http://jama.jamanetwork.com/Issue.aspx?journalid=67&issueID=4452&direction=P. Click the new bookmarklet you got on your toolbar. The page will reload and you will be asked to log in through the Florida International University EZproxy system. When you are authenticated, you will be seeing the page proxied: http://jama.jamanetwork.com.ezproxy.fiu.edu/Issue.aspx?journalid=67&issueID=4452&direction=P.

You will be surprised to see how simple the bookmarklet is (and there is even a shorter version than this which I will show in the next section). It is a JavaScript function wrapped inside a hyperlink. Lines 2-5 each takes the domain name, the path name, and any search string after the url path from the current window location object. So in the case of http://jama.jamanetwork.com.ezproxy.fiu.edu/Issue.aspx?journalid=67&issueID=4452&direction=P, location.host is http://jama.jamanetwork.com and location.pathname is Issue.aspx . The rest of the url ?journalid=67&issueID=4452&direction=P — is location.search. In line 4, I am putting my institution’s ezproxy suffix between these two, and in line 5, I am asking the browser load this new link.

<a href="javascript:(function(){
	var host=location.host;
	var path=location.pathname;
	var srch=location.search;
	var newlink='http://'+host+'.ezproxy.fiu.edu'+path+srch;
	window.open(newlink);
})();">Full-Text from FIU Library</a>

Now let’s take a look at the whole form. I created this form for those who want to create a ready-made bookmarklet recipe. All they need is their institution’s EZproxy suffix and whatever name they want to give to the bookmarklet. Once one fills out those two fields and click ‘Customize’ button, one will get the full HTML page code with the bookmarklet as a link in it.

<html>
<head>
<script type="text/javascript" src="http://ajax.googleapis.com/ajax/libs/jquery/1.8.0/jquery.min.js"></script>
</head>
<body>
<h1>EZproxy Bookmarklet</h1>	
<p><ul><li>
	<a href="javascript:(function(){
		var host=location.host;
		var path=location.pathname;
                var srch=location.search;
                var newlink='http://'+host+'.ezproxy.fiu.edu'+path+srch;
		window.open(newlink);
	})();">Full-Text from FIU Library</a> 
	(Drag the link to the bookmark toolbar!)
</li></ul></p>
<p>This is a bookmarket that will reroute a current page through FIU EZproxy.
	<br/>
If you have the LibX toolbark installed, you do not need this bookmarklet. Simply select 'Reload Page via XYZ EZproxy' on the menu that appears when you right click.
	<br/>
Created by Bohyun Kim on March, 2013. 	
</p>
<h2>How to Test</h2>
<ul>
	<li>Drag the link above to the bookmark toolbar of your web browser.</li>
	<li>Click the bookmark when you are on a webpage that has an academic article.</li>
	<li>It will ask you to log in through the FIU EZproxy.</li>
	<li>Once you are authenticated and the library has a subscription to the journal, you will will able to get the full-text article on the page.</li>
	<li>Look at the url and see if it contains .ezproxy.fiu.edu. If it does, the bookmarklet is working.</li>
</ul>	

<h2>Make One for Your Library</h2>
	<p>
		Bookmark title: <input type="text" id="title1" size="40" placeholder="e.g. Full-Text ABC Library"/>
		<em>e.g. Full-text ABC Library</em>
		<br/>
		Library EZproxy Suffix: <input type="text" id="link1" size="31" placeholder="e.g. ezproxy.abc.edu"/>
		<em>e.g. ezproxy.abc.edu
		<br/>
		<button id="b1">Customize</button></em>
	</p>
	<p><strong>Copy the following into a text editor and save it as an html file.</strong></p>
	<ul>
		<li>Open the file in a web browser and drag your link to the bookmark toolbar.</li>
	</ul>	
	<p id="result1" style="color:#F7704F;">**Customized code will appear here.**</p>
	<p><strong>If you want to make any changes to the code:</strong></p>
	<textarea rows="10" cols="60"></textarea>

	<script type="text/javascript">
	$(document).ready(function(){	
	  $("#b1").click(function(){
		var pre="&lt;html&gt; &lt;head&gt; &lt;script type=&quot;text/javascript&quot; src=&quot;http://ajax.googleapis.com/ajax/libs/jquery/1.8.0/jquery.min.js&quot;&gt;&lt;/script&gt; &lt;/head&gt; &lt;body&gt; &lt;a href=&quot;javascript:(function(){ var hst=location.host; var path=location.pathname; var srch=location.search; var newlink='http://'+hst";
		var link1=$('#link1').val();
		var post="'+path+srch; window.open(newlink); })();">";
	  	var title=$("#title1").val();
	  	var end="&lt;/a&gt; &lt;/body&gt; &lt;/html&gt;";
	  	var final=$('<div />').html(pre+"+'."+link1+post+title+end).text()
	  	$('#result1').text(final);
	    $('textarea').val(final);
	  });
	}); // doc ready
	</script>

</body>
</html>

That ‘customize’ part is done here with several lines of JQuery. You will notice that the process is quite similar to what I did in my first example. In lines 4-8, I am just stitching together a bunch of text strings to spit out the whole code in the text area eventually when the ‘Customize’ button is clicked. All special characters used in HTML tags such as ‘<‘ and ‘>’ have been changed to html enities. In line 9, I am taking the entire string saved in the variable end —I hope you name your variables a little more carefully than I do!–  and adding it to an empty div so that the string would be set as the inner HTML property of that div. And then I retrieve it using the .text() method. The result is the HTML code with the html entities decoded.

<script type="text/javascript">
	$(document).ready(function(){	
	  $("#b1").click(function(){
		var pre="&lt;html&gt; &lt;head&gt; &lt;script type=&quot;text/javascript&quot; src=&quot;http://ajax.googleapis.com/ajax/libs/jquery/1.8.0/jquery.min.js&quot;&gt;&lt;/script&gt; &lt;/head&gt; &lt;body&gt; &lt;a href=&quot;javascript:(function(){ var hst=location.host; alert(hst); var path=location.pathname; var srch=location.search; var newlink='http://'+hst";
		var link1=$('#link1').val();
		var post="'+path+srch; window.open(newlink); })();">";
	  	var title=$("#title1").val();
	  	var end="&lt;/a&gt; &lt;/body&gt; &lt;/html&gt;";
	  	var final=$('<div />').html(pre+"+'."+link1+post+title+end).text()
	  	$('#result1').text(final);
	    $('textarea').val(final);
	  });
	}); // doc ready
</script>

Not too bad, right? I hope these examples show how just a few or several lines of code can be used to solve your practical problems at work. Coding is there for you to automate time-consuming and/or repetitive tasks.

3. EZproxy bookmarklet generator – Shorter version

There is a simpler way to create a EZproxy bookmarklet than the one sketched above. If you simply add EZproxy prefix in front of the entire url of the page where a user is is, you achieve the same result. In this case, you do not have to break the url with host, pathname, search string, etc.

<a href="javascript:void(location.href=%22http://ezproxy.fiu.edu/login?url=%22+location.href)">Full-Text from FIU Library</a>

Here are the code for this much simpler EZproxy bookmarklet and the bookmarklet generator. If you know the prefix of you library’s EZproxy prefix, you can make one for your library.

So there are many ways to get the same thing done. Some are more elegant and some are less so. Usually a shorter one is a more elegant solution. The lesson is that you usually get to the most elegant solution after coming up with many less elegant solutions first.

Do you have a problem that can be solved by creating several lines of code? Have you ever solved a practical problem using a bit of code? Share your experience in the comments!

 

Aaron Swartz and Too-Comfortable Research Libraries

*** This post has been originally published in ACRL TechConnect on Feb. 11, 2013. ***
*** Update: Several references and a video added (thanks to Brett Bonfield) on Feb. 21, 2013. ***

Who was Aaron Swartz?

If you are a librarian and do not know who Aaron Swartz is, that should probably change now. He helped developing the RSS standard, was the co-founder of Reddit, worked on the Open Library project, downloaded and freed 20% (2.7 million documents) of the Public Access to Court Electronic Records (PACER) database that charges fees for the United States federal court documents, out of which about 1,600 had privacy issues, played a lead role in preventing the Stop Online Piracy Act (SOPA), and wrote the Guerrilla Open Access Manifesto.

Most famously, he was arrested in 2011 for the mass download of journal articles from JSTOR.  He returned the documents to JSTOR and apologized. The Massachusetts state court dismissed the charges, and JSTOR decided not to pursue civil litigation. But MIT stayed silent, and the federal court charged Swartz with wire fraud, computer fraud, unlawfully obtaining information from a protected computer and recklessly damaging a protected computer. If convicted on these charges, Swartz could be sentenced to up to 35 years in prison at the age of 26. He committed suicide after facing charges for two years, on January 11, 2013.

Information wants to be free; Information wants to be expensive

Now, he was a controversial figure. He advocated Open Access (OA) but to the extent of encouraging scholars, librarians, students who have access to copyrighted academic materials to trade passwords and circulate them freely on the grounds that this is an act of civil disobedience against unjust copyright laws in his manifesto. He was an advocate of the open Internet, the transparent government, and open access to scholarly output. But he also physically hacked into the MIT network wiring closet and attached his laptop to download over 4 million articles from JSTOR. Most people including librarians are not going to advocate trading their institutions’ subscription database passwords or breaking into a staff-only computer networking area of an institution. The actual method of OA that Swartz recommended was highly controversial even among the strongest OA advocates.

But in his Guerrilla OA manifesto, Swartz raised one very valid point about the nature of information in the era of the World Wide Web. That is, information is power. (a) As power, information can be spread to and be made useful to as many of us as possible. Or, (b) it can be locked up and the access to it can be restricted to only those who can pay for it or have access privileges some other way. One thing is clear. Those who do not have access to information will be at a significant disadvantage compared to those who do.

And I would like to ask what today’s academic and/or research libraries are doing to realize Scenario (a) rather than Scenario (b). Are academic/research libraries doing enough to make information available to as many as possible?

Too-comfortable Internet, Too-comfortable academic libraries

Among the many articles I read about Aaron Swartz’s sudden death, the one that made me think most was “Aaron Swartz’s suicide shows the risk of a too-comfortable Internet.” The author of this article worries that we may now have a too-comfortable Internet. The Internet is slowly turning into just another platform for those who can afford purchasing information. The Internet as the place where you could freely find, use, modify, create, and share information is disappearing. Instead pay walls and closed doors are being established. Useful information on the Internet is being fast monetized, and the access is no longer free and open. Even the government documents become no longer freely accessible to the public when they are put up on the Internet (likely to be due to digitization and online storage costs) as shown in the case of PACER and Aaron Swartz. We are more and more getting used to giving up our privacy or to paying for information. This may be inevitable in a capitalist society, but should the same apply to libraries as well?

The thought about the too-comfortable Internet made me wonder whether perhaps academic research libraries were also becoming too comfortable with the status quo of licensing electronic journals and databases for patrons. In the times when the library collection was physical, people who walk into the library were rarely turned away. The resources in the library are collected and preserved because we believe that people have the right to learn and investigate things and to form one’s own opinions and that the knowledge of the past should be made available for that purpose. Regardless of one’s age, gender, social and financial status, libraries have been welcoming and encouraging people who were in the quest for knowledge and information.  With the increasing number of electronic resources in the library, however, this has been changing.

Many academic libraries offer computers, which are necessary to access electronic resources of the library itself. But how many of academic libraries keep all the computers open for user without the user log-in? Often those library computers are locked up and require the username and password, which only those affiliated with the institution possess. The same often goes for many electronic resources. How many academic libraries allow the on-site access to electronic resources by walk-in users? How many academic libraries insist on the walk-in users’ access to those resources that they pay for in the license? Many academic libraries also participate in the Federal Depository Library program, which requires those libraries to provide free access to the government documents that they receive to the public. But how easy is it for the public to enter and access the free government information at those libraries?

I asked in Twitter about the guest access in academic libraries to computers and e-resources. Approximately 25 academic librarians generously answered my question. (Thank you!) According to the responses in Twitter,  almost all except a few libraries ( mentioned in Twitter responses) offer guest access to computers and e-resources on-site. It is to be noted, however, that a few offer the guest -access to neither. Also some libraries limit the guests’ computer-use to 30 minutes – 4 hours, thereby restricting the access to the library’s electronic resources as well. Only a few libraries offer free wi-fi for guests. And at some libraries, the guest wi-fi users are unable to access the library’s e-resources even on-site because the IP range of the guest wi-fi is different from that of the campus wi-fi.

I am not sure how many academic libraries consciously negotiate the walk-in users’ on-site access with e-resources vendors or whether this is done somewhat semi-automatically because many libraries ask the library building IP range to be registered with vendors so that the authentication can be turned off inside the building. I surmise that publishers and database vendors will not automatically permit the walk-in users’ on-site access in their licenses unless libraries ask for it. Some vendors also explicitly prohibit libraries from using their materials to fill the Interlibrary loan requests from other libraries. The electronic resource vendors and publishers’ pricing has become more and more closely tied to the number of patrons who can access their products. Academic libraries has been dealing with the escalating costs for electronic resources by filtering out library patrons and limiting the access to those in a specific disciplines.  For example, academic medical and health sciences libraries often subscribe to databases and resources that have the most up-to-date information about biomedical research, diseases, medications, and treatments. These are almost always inaccessible to the general public and often even to those affiliated with the institution. The use of these prohibitively expensive resources is limited to a very small portion of people who are affiliated with the institution in specific disciplines such as medicine and health sciences. Academic research libraries have been partially responsible for the proliferation of these access limitations by welcoming and often preferring these limitations as a cost-saving measure. (By contrast, if those resources were in the print format, no librarian would think that it is OK to permanently limit its use to those in medical or health science disciplines only.)

Too-comfortable libraries do not ask themselves if they are serving the public good of providing access to information and knowledge for those who are in need but cannot afford it. Too-comfortable libraries see their role as a mediator and broker in the transaction between the information seller and the information buyer. They may act as an efficient and successful mediator and broker. But I don’t believe that that is why libraries exist. Ultimately, libraries exist to foster the sharing and dissemination of knowledge more than anything, not to efficiently mediate information leasing.  And this is the dangerous idea: You cannot put a price tag on knowledge; it belongs to the human race. Libraries used to be the institution that validates and confirms this idea. But will they continue to be so in the future? Will an academic library be able to remain as a sanctuary for all ideas and a place for sharing knowledge for people’s intellectual pursuits regardless of their institutional membership? Or will it be reduced to a branch of an institution that sells knowledge to its tuition-paying customers only? While public libraries are more strongly aligned with this mission of making information and knowledge freely and openly available to the public than academic libraries, they cannot be expected to cover the research needs of patrons as fully as academic libraries.

I am not denying that libraries are also making efforts in continuing the preservation and access to the information and resources through initiatives such as Hathi Trust and DPLA (Digital Public Library of America). My concern is rather whether academic research libraries are becoming perhaps too well-adapted to the times of the Internet and online resources and too comfortable serving the needs of the most tangible patron base only in the most cost-efficient way, assuming that the library’s mission of storing and disseminating knowledge can now be safely and neutrally relegated to the Internet and the market. But it is a fantasy to believe that the Internet will be a sanctuary for all ideas (The Internet is being censored as shown in the case of Tarek Mehanna.), and the market will surely not have the ideal of the free and open access to knowledge for the public.

If libraries do not fight for and advocate those who are in need of information and knowledge but cannot afford it, no other institution will do so. Of course, it costs to create, format, review, and package content. Authors as well as those who work in this business of content formatting, reviewing, packaging, and producing should be compensated for their work. But not to the extent that the content is completely inaccessible to those who cannot afford to purchase but nevertheless want access to it for learning, inquiry, and research. This is probably the reason why we are all moved by Swartz’s Guerrilla Open Access Manifesto in spite of the illegal implications of the action that he actually recommended in the manifesto.

Knowledge and information is not like any other product for purchase. Sharing increases its value, thereby enabling innovation, further research, and new knowledge. Limiting knowledge and information to only those with access privilege and/or sufficient purchasing power creates a fundamental inequality. The mission of a research institution should never be limited to self-serving its members only, in my opinion. And if the institution forgets this, it should be the library that first raises a red flag. The mission of an academic research institution is to promote the freedom of inquiry and research and to provide an environment that supports that mission inside and outside of its walls, and that is why a library is said to be the center of an academic research institution.

I don’t have any good answers to the inevitable question of “So what can an academic research library do?” Perhaps, we can start with broadening the guest access to the library computers, wi-fi, and electronic resources on-site. Academic research libraries should also start asking themselves this question: What will libraries have to offer for those who seek knowledge for learning and inquiry but cannot afford it? If the answer is nothing, we will have lost libraries.

In his talk about the Internet Archive’s Open Library project at the Code4Lib Conference in 2008 (at 11:20), Swartz describes how librarians had argued about which subject headings to use for the books in the Open Library website. And he says, “We will use all of them. It’s online. We don’t have to have this kind of argument.” The use of online information and resources does not incur additional costs for use once produced. Many resources, particularly those scholarly research output already have established buyers such as research libraries. Do we have to deny access to information and knowledge to those who cannot afford but are seeking for it, just so that we can have a market where information and knowledge resources are sold and bought and authors are compensated along with those who work with the created content as a result? No, this is a false question. We can have both. But libraries and librarians will have to make it so.

Videos to Watch


“Code4Lib 2008: Building the Open Library – YouTube.”

“Aaron Swartz on Picking Winners” American Library Association Midwinter meeting, January 12, 2008.


“Freedom to Connect: Aaron Swartz (1986-2013) on Victory to Save Open Internet, Fight Online Censors.”

REFERENCES

“Aaron Swartz.” 2013. Accessed February 10. http://www.aaronsw.com/.

“Aaron Swartz – Wikipedia, the Free Encyclopedia.” 2013. Accessed February 10. http://en.wikipedia.org/wiki/Aaron_Swartz#JSTOR.

“Aaron Swartz on Picking Winners – YouTube.” 2008. http://www.youtube.com/watch?feature=player_embedded&v=BvJqXaoO4FI.

“Aaron Swartz’s Suicide Shows the Risk of a Too-comfortable Internet – The Globe and Mail.” 2013. Accessed February 10. http://www.theglobeandmail.com/commentary/aaron-swartzs-suicide-shows-the-risk-of-a-too-comfortable-internet/article7509277/.

“Academics Remember Reddit Co-Founder With #PDFTribute.” 2013. Accessed February 10. http://www.slate.com/blogs/the_slatest/2013/01/14/aaron_swartz_death_pdftribute_hashtag_aggregates_copyrighted_articles_released.html.

“After Aaron, Reputation Metrics Startups Aim To Disrupt The Scientific Journal Industry | TechCrunch.” 2013. Accessed February 10. http://techcrunch.com/2013/02/03/the-future-of-the-scientific-journal-industry/.

American Library Association, “A Memorial Resolution Honoring Aaron Swartz.” 2013. http://connect.ala.org/files/memorial_5_aaron%20swartz.pdf.

“An Effort to Upgrade a Court Archive System to Free and Easy – NYTimes.com.” 2013. Accessed February 10. http://www.nytimes.com/2009/02/13/us/13records.html?_r=1&.

Bonfield, Brett. 2013. “Aaron Swartz.” In the Library with the Lead Pipe (February 20). http://www.inthelibrarywiththeleadpipe.org/2013/aaron-swartz/.

“Code4Lib 2008: Building the Open Library – YouTube.” 2013. Accessed February 10. http://www.youtube.com/watch?v=oV-P2uzzc4s&feature=youtu.be&t=2s.

“Daily Kos: What Aaron Swartz Did at MIT.” 2013. Accessed February 10. http://www.dailykos.com/story/2013/01/13/1178600/-What-Aaron-Swartz-did-at-MIT.

Dupuis, John. 2013a. “Around the Web: Aaron Swartz Chronological Link Roundup – Confessions of a Science Librarian.” Accessed February 10. http://scienceblogs.com/confessions/2013/01/20/around-the-web-aaron-swartz-chronological-link-roundup/.

———. 2013b. “Library Vendors, Politics, Aaron Swartz, #pdftribute – Confessions of a Science Librarian.” Accessed February 10. http://scienceblogs.com/confessions/2013/01/17/library-vendors-politics-aaron-swartz-pdftribute/.

“FDLP for PUBLIC.” 2013. Accessed February 10. http://www.gpo.gov/libraries/public/.

“Freedom to Connect: Aaron Swartz (1986-2013) on Victory to Save Open Internet, Fight Online Censors.” 2013. Accessed February 10. http://www.democracynow.org/2013/1/14/freedom_to_connect_aaron_swartz_1986.

“Full Text of ‘Guerilla Open Access Manifesto’.” 2013. Accessed February 10. http://archive.org/stream/GuerillaOpenAccessManifesto/Goamjuly2008_djvu.txt.

Groover, Myron. 2013. “British Columbia Library Association – News – The Last Days of Aaron Swartz.” Accessed February 21. http://www.bcla.bc.ca/page/news/ezlist_item_9abb44a1-4516-49f9-9e31-57685e9ca5cc.aspx#.USat2-i3pJP.

Hellman, Eric. 2013a. “Go To Hellman: Edward Tufte Was a Proto-Phreaker (#aaronswnyc Part 1).” Accessed February 21. http://go-to-hellman.blogspot.com/2013/01/edward-tufte-was-proto-phreaker.html.

———. 2013b. “Go To Hellman: The Four Crimes of Aaron Swartz (#aaronswnyc Part 2).” Accessed February 21. http://go-to-hellman.blogspot.com/2013/01/the-four-crimes-of-aaron-swartz.html.

“How M.I.T. Ensnared a Hacker, Bucking a Freewheeling Culture – NYTimes.com.” 2013. Accessed February 10. http://www.nytimes.com/2013/01/21/technology/how-mit-ensnared-a-hacker-bucking-a-freewheeling-culture.html?pagewanted=all.

March, Andrew. 2013. “A Dangerous Mind? – NYTimes.com.” Accessed February 10. http://www.nytimes.com/2012/04/22/opinion/sunday/a-dangerous-mind.html?pagewanted=all.

“MediaBerkman » Blog Archive » Aaron Swartz on The Open Library.” 2013. Accessed February 22. http://blogs.law.harvard.edu/mediaberkman/2007/10/25/aaron-swartz-on-the-open-library-2/.

Peters, Justin. 2013. “The Idealist.” Slate, February 7. http://www.slate.com/articles/technology/technology/2013/02/aaron_swartz_he_wanted_to_save_the_world_why_couldn_t_he_save_himself.html.

“Public Access to Court Electronic Records.” 2013a. Accessed February 10. http://www.pacer.gov/.

“Publishers and Library Groups Spar in Appeal to Ruling on E-Reserves – Technology – The Chronicle of Higher Education.” 2013. Accessed February 10. http://chronicle.com/article/PublishersLibrary-Groups/136995/?cid=pm&utm_source=pm&utm_medium=en.

“Remember Aaron Swartz.” 2013. Celebrating Aaron Swartz. Accessed February 22. http://www.rememberaaronsw.com.

Rochkind, Jonathan. 2013. “Library Values and the Growing Scholarly Digital Divide: In Memoriam Aaron Swartz | Bibliographic Wilderness.” Accessed February 10. http://bibwild.wordpress.com/2013/01/13/library-values-and-digital-divide-in-memoriam-aaron-swartz/.

Sims, Nancy. 2013. “What Is the Government’s Interest in Copyright? Not That of the Public. – Copyright Librarian.” Accessed February 10. http://blog.lib.umn.edu/copyrightlibn/2013/02/what-is-the-governments-interest-in-copyright.html.

Stamos, Alex. 2013. “The Truth About Aaron Swartz’s ‘Crime’.” Unhandled Exception. Accessed February 22. http://unhandled.com/2013/01/12/the-truth-about-aaron-swartzs-crime/.

Summers, Ed. 2013. “Aaronsw | Inkdroid.” Accessed February 21. http://inkdroid.org/journal/2013/01/19/aaronsw/.

“The Inside Story of Aaron Swartz’s Campaign to Liberate Court Filings | Ars Technica.” 2013. Accessed February 10. http://arstechnica.com/tech-policy/2013/02/the-inside-story-of-aaron-swartzs-campaign-to-liberate-court-filings/.

“Welcome to Open Library (Open Library).” 2013. Accessed February 10. http://openlibrary.org/.

West, Jessamyn. 2013. “Librarian.net » Blog Archive » On Leadership and Remembering Aaron.” Accessed February 21. http://www.librarian.net/stax/3984/on-leadership-and-remembering-aaron/.

 

More APIs: writing your own code (2)

*** This blog post has been originally published in ACRL TechConnect 0n October 9, 2012. ***

My previous post “The simpest AJAX: writing your own code (1)” discussed a few Javascript and JQuery examples that make the use of the Flickr API. In this post, I try out APIs from providers other than Flickr. The examples will look plain to you since I didn’t add any CSS to dress them up. But remember that the focus here is not in presentation but in getting the data out and re-displaying it on your own. Once you get comfortable with this process, you can start thinking about a creative and useful way in which you can present and mash up the same data. We will go through 5 examples I created with three different APIs.  Before taking a look at the codes, check out the results below first.

I. Pinboard API

The first example is Pinboard. Many libraries moved their bookmarks in Del.icio.us to a different site when there was a rumor that Del.cio.us may be shut down by Yahoo. One of those sites were Pinboard.  By getting your bookmark feeds from Pinboard and manipulating them, you can easily present a subset of your bookmark as part of your website.

(a) Display bookmarks in Pinboard using its API

The following page uses JQuery to access the JSONP feed of my public bookmarks in Pinboard. $.ajax() method is invoked on line 13. Line 15, jsonp:”cb”, gives the name to a callback function that will wrap the JSON feed data in it. Note line 18 where I print out data received into the console. This way, you can check if you are receiving JSONP feed in the console of Firebug. Line 19-22 uses $.each() function to access each element in the JSONP feed and the .append() method to add each bookmark’s title and url to the “pinboard” div. JQuery documentation has detailed explanation and examples for its functions and methods. So make sure to check it out if you have any questions about a JQuery function or method.

Pinboard API – example 1

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>Pinboard: JSONP-AJAX Example</title>
<script type="text/javascript" src="http://jqueryjs.googlecode.com/files/jquery-1.3.2.min.js"></script>
</head>
<body>
<p> This page takes the JSON feed from <a href="http://pinboard.in/u:bohyunkim/">my public links in Pinboard</a> and re-presents them here. See <a href="http://feeds.pinboard.in/json/u:bohyunkim/">the Pinboard's JSON feed of my links</a>.</p>
<div id="pinboard"><h1>All my links in Pinboard</h1></div>
<script>
$(document).ready(function(){	
$.ajax({
  url:'http://feeds.pinboard.in/json/u:bohyunkim',
  jsonp:"cb",
  dataType:'jsonp',
  success: function(data) {
  	console.log(data); //dumps the data to the console to check if the callback is made successfully
    $.each(data, function(index, item){
      $('#pinboard').append('<div><a href="' + item.u + '">' + item.d
+ '</a></div>');
      }); //each
    } //success
  }); //ajax

});//ready
</script>
</body>
</html>

Here is the screenshot of the page. I opened up the Console window of the Firebug (since I dumped the received in line 18) and you can see the incoming data here. (Note. Click the images to see the large version.)

But it is much more convenient to see the organization and hierarchy of the JSONP feed in the Net panel of Firebug.

And each element of the JSONP feed can be drilled down for further details by clicking the object in each row.

(b) Display only a certain number of bookmarks

Now, let’s display only five bookmarks. In order to do this, only one more line is needed. Line 9 checks the position of each element and breaks the loop when the 5th element is processed.

Pinboard API – example 2

$.ajax({
  url:'http://feeds.pinboard.in/json/u:bohyunkim',
  jsonp:"cb",
  dataType:'jsonp',
  success: function(data) {
    $.each(data, function(index, item){
      	$('#pinboard').append('<div><a href="' + item.u + '">' + item.d
+ '</a></div>');
    	if (index == 4) return false; //only display 5 items
      }); //each
    } //success
  }); //ajax

(c) Display bookmarks with a certain tag

Often libraries want to display bookmarks with a particular tag. Here I add a line using JQuery method $.inArray() to display only bookmarks tagged with ‘fiu.’ $.inArray()  method takes value and array as parameters and returns 0 if the value is found in the array otherwise -1. Line 7 checks if the tag array of a bookmark (item.t) does include ‘fiu,’ and only in such case displays the bookmark. As a result, only the bookmarks with the tag ‘fiu’ are shown in the page.

Pinboard API – example 3

$.ajax({
  url:'http://feeds.pinboard.in/json/u:bohyunkim',
  jsonp:"cb",
  dataType:'jsonp',
  success: function(data) {
    $.each(data, function(index, item){
    	if ($.inArray("fiu", item.t)!==-1) // if the tag is 'fiu'
      		$('#pinboard').append('<div><a href="' + item.u + '">' + item.d
+ '</a></div>');
	}); //each
    } //success
  }); //ajax

II. Reddit API

My second example uses Reddit API. Reddit is a site where people comment on news items of interest. Here I used $.getJSON() instead of $.ajax() in order to process the JSONP feed from the Science section of Reddit. In the case of Pinboard API, I could not find out a way to construct a link that includes a call back function in the url. Some of the parameters had to be specified such as jsonp:”cb”, dataType:’jsonp’. For this reason, I needed to use $.ajax() function. On the other hand, in Reddit, getting the JSONP feed url was straightforward: http://www.reddit.com/r/science/.json?jsonp=cb.

Line 19 adds a title of the page. Line 20-22 extracts the title and link to the news article that is being commented and displays it. Under the news item, the link to the comments for that article in Reddit is added as a bullet item. You can see that, in Line 17 and 18, I have used the console to check if I get the right data and targeting the element I want and then commented out later.

This is just an example, and for that reason, the result is a rather simplified version of the original Reddit page with less information. But as long as you are comfortable accessing and manipulating data at different levels of the JSONP feed sent from an API, you can slice and dice the data in a way that suits your purpose best. So in order to make a clever mash-up, not only the technical coding skills but also your creative ideas of what different sets of data and information to connect and present to offer something new that has not been available or apparent before.

My second example uses Reddit API. Reddit is a site where people comment on news items of interest. Here I used $.getJSON() method instead of $.ajax() in order to process the JSONP feed from the Science section of Reddit. In the case of Pinboard API, I could not find out a way to construct a link for a call back function. Some of the parameters had to be specified such as jsonp:”cb”, dataType:’jsonp’. So I needed to use $.ajax() method.

On the other hand, in Reddit, getting the JSONP feed url was straightforward: http://www.reddit.com/r/science/.json?jsonp=cb.   Line 19 adds a title of the page. Line 20-22 extracts the title and link to the news article that is being commented and displays it. Under the news item, the link to the comments for that article in Reddit is added as a bullet item.You can see that in Line 17 and 18, I have used the console to check if I get the right data and targeting the element I want.

This is just an example, and for that reason, the result is rather a simplified version of the original Reddit page with less information. But as long as you are comfortable accessing and manipulating data at different levels of the JSONP feed sent from an API, you can slice and dice the data in a way that suits your purpose best.

Reddit API – example

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>Reddit-Science: JSONP-AJAX Example</title>
<script type="text/javascript" src="http://ajax.googleapis.com/ajax/libs/jquery/1.8.0/jquery.min.js"></script>
</head>

<body>
<p> This page takes the JSONP feed from <a href="http://www.reddit.com/r/science/">Reddit's Science section</a> and presents the link to the original article and the comments in Reddit to the article as a pair. See <a href="http://www.reddit.com/r/science/.json?jsonp=?">JSONP feed</a> from Reddit.</p>

<div id="feed"> </div>
<script type="text/javascript">
	//run function to parse json response, grab title, link, and media values, and then place in html tags
$(document).ready(function(){		
	$.getJSON('http://www.reddit.com/r/science/.json?jsonp=?', function(rd){
		//console.log(rd);
		//console.log(rd.data.children[0].data.title);
		$('#feed').html('<h1>*Subredditum Scientiae*</h1>');
		$.each(rd.data.children, function(k, v){
      		$('#feed').append('<div><p>'+(k+1)+': <a href="' + v.data.url + '">' + v.data.title+'</a></p><ul><li style="font-variant:small-caps;font-size:small"><a href="http://www.reddit.com'+v.data.permalink+'">Comments from Reddit</a></li></ul></div>');
      }); //each
	}); //getJSON
});//ready	
</script>

</body>
</html>

The structure of a JSON feed can be confusing to make out particularly. So make sure to use the Firebug Net window to figure out the organization of the feed content and the property name for the value you want.

But what if the site from which you would like to get data doesn’t offer JSONP feed? Fortunately you can convert any RSS or XML feed into JSONP feed. Let’s take a look!

III. PubMed Feed with Yahoo Pipes API

Consider this PubMed search. This is simple search that looks for items in PubMed that has to do with Florida International University College of Medicine where I work. You may want to access the data feed of this search result, manipulate, and display in your library website. So far, we have performed a similar task with the Pinboard and the Reddit API using JQuery. But unfortunately PubMed does not offer any JSON feed. We only get RSS feed instead from PubMed.

This is OK, however. You can either manipulate the RSS feed directly or convert the RSS feed into JSON, which you are more familiar with now. Yahoo Pipes is a handy tool for just that purpose. You can do the following tasks with Yahoo Pipes:

  • combine many feeds into one, then sort, filter and translate it.
  • geocode your favorite feeds and browse the items on an interactive map.
  • power widgets/badges on your web site.
  • grab the output of any Pipes as RSS, JSON, KML, and other formats.

Furthermore, there may be a pipe that has been already created for exactly what you want to do by someone else. As PubMed is a popular resource, I found a pipe for PubMed search. I tested, copied the pipe, and changed the search term. Here is the screenshot of my Yahoo Pipe.

If you want to change the pipe, you can click “View Source” and make further changes. Here I just changed the search terms and saved the pipe.

After that, you want to get the results of the Pipe as JSON. If you hover over the “Get as JSON” link in the first screenshot above, you will get a link: http://pipes.yahoo.com/pipes/pipe.run?_id=e176c4da7ae8574bfa5c452f9bb0da92&_render=json&limit=100&term=”Florida International University” and “College of Medicine” But this returns JSON, not JSONP.

In order to get that JSON feed wrapped into a callback function, you need to add this bit, &_callback=myCallback, at the end of the url: http://pipes.yahoo.com/pipes/pipe.run?_id=e176c4da7ae8574bfa5c452f9bb0da92&_render=json&limit=10&term=%22Florida+International+University%22+and+%22College+of+Medicine%22&_callback=myCallback. Now the JSON feed appears wrapped in a callback function like this: myCallback( ). See the difference?

Line 25 enables you to bring in this JSONP feed and invokes the callback function named “myCallback.” Line 14-23 defines this callback function to process the received feed. Line 18-20 takes the JSON data received at the level of data.value. item, and prints out each item’s title (item.title) with a link (item.link). Here I am giving a number for each item by (index+1). If you don’t put +1, the index will begin from 0 instead of 1. Line 21 stops the process when the processed item reaches 5 in number.

Yahoo Pipes API/PubMed – example

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>PubMed and Yahoo Pipes: JSONP-AJAX Example</title>
<script type="text/javascript" src="http://jqueryjs.googlecode.com/files/jquery-1.3.2.min.js"></script>
</head>
<body>
<p> This page takes the JSONP feed from <a href="http://pipes.yahoo.com/pipes/pipe.info?_id=e176c4da7ae8574bfa5c452f9bb0da92"> a Yahoo Pipe</a>, which creates JSONP feed out of a PubMed search results and re-presents them here. 
<br/>See <a href="http://pipes.yahoo.com/pipes/pipe.run?_id=e176c4da7ae8574bfa5c452f9bb0da92&_render=json&limit=10&term=%22Florida+International+University%22+and+%22College+of+Medicine%22&_callback=myCallback">the Yahoo Pipe's JSON feed</a> and <a href="http://www.ncbi.nlm.nih.gov/pubmed?term=%22Florida%20International%20University%22%20and%20%22College%20of%20Medicine%22">the original search results in PubMed</a>.</p>
<div id="feed"></div>
<script type="text/javascript">
	//run function to parse json response, grab title, link, and media values - place in html tags
	function myCallback(data) {
		//console.log(data);
		//console.log(data.count);
		$("#feed").html('<h1>The most recent 5 publications from <br/>Florida International University College of Medicine</h1><h2>Results from PubMed</h2>');
		$.each(data.value.items, function(index, item){
      		$('#feed').append('<p>'+(index+1)+': <a href="' + item.link + '">' + item.title
+ '</a></p>');
        if (index == 4) return false; //display the most recent five items
      }); //each
	} //function
	</script>
<script type="text/javascript" src="http://pipes.yahoo.com/pipes/pipe.run?_id=e176c4da7ae8574bfa5c452f9bb0da92&_render=json&limit=10&term=%22Florida+International+University%22+and+%22College+of+Medicine%22&_callback=myCallback"></script>
</body>
</html>

Do you feel more comfortable now with APIs? With a little bit of JQuery and JSON, I was able to make a good use of third-party APIs. Next time, I will try the Worldcat Search API, which is closer to the library world and see how that works.

A bad business model or a savvy market strategy? Thinking like a vendor

I want to talk about an e-book platform called Inkling whose web version is not cross-browser compatible. But what I am really interested to talk about is neither an ebook platform nor its cross-browser support. I am interested in thinking about the way we assess and evaluate resource products from vendors and their market strategies as librarians. I am using Inkling just as an example to touch on this topic.

Inkling is a company that makes an interactive textbook. It works with publishers to get a contract for certain titles of theirs, so that those titles can be made to e-books that run on and are sold at the Inkling platform. Inkling originally started as an e-book platform app on the iOS device such as an iPad and iPhone. The Inkling platform is quite nice. Adding annotation is easy and the page numbers of a print book is clearly marked on the side of its e-book version. Inkling e-textbooks in medicine offer interactive quizzes integrated into a human anatomy diagram and include related audio and video files that can be played right there inside the app, thereby broadening the utility of a textbook to students. The Inkling e-book platform also provides various sharing features that can be handy for students and teachers.

Recently, Inkling released a Web version. My library, which tested its app version earlier, was interested in purchasing some of the titles that were available at the Inkling Store for library patrons. Not only did we like the Inkling platform as a reading platform but we also found out that those titles were not available as e-books from any other vendors. My library wanted to provide more medical titles as e-books so that our students and faculty can access them with convenience. But while testing their Web version, we found a few problematic things for library patrons. (1) First, the Web version of Inkling doesn’t allow library patrons to use their iPad/iPhone version unlike products such as Naxos Music Library. So the convenience of getting the resource on a personal device was taken away in the Web version. (2) Second, Inkling requires each library patron to create a separate Inkling account in order to access the Web version of Inkling through the institutional subscription. The double authorization required – one for the university log-in and the other for the inkling log-in – was cumbersome and annoying. The majority of library users do not want to create an additional account for each database or resource that they use through the library subscription.

But the most problematic was (3) the fact that Inkling’s Web version only runs on Safari and Chrome.  No Firefox, no IE. This was a deal-breaker for us because the number one browser that is used to access our library’s website is IE and the second is Firefox according to the Google Analytics statistics. Furthermore, the most important patron group that my library serves, the medical students and faculty at our school, mostly work on the student laptops and office PCs that the school IT department configures and issues, and those student laptops and computers only come with IE and Firefox. Students and faculty have to request IT if they want to install any new software on these standard laptops and PCs. For this reason, when the limited browser support of the Inkling Web was known, librarians at my library unanimously agreed to wait for Inkling to add the support for other web browsers. We also did a demo of the Web version to a group of faculty with a few students included but the responses were lukewarm. So waiting was agreed to be the best option at this point.

At the same time, it was puzzling why Inkling released a Web product that only supports Safari and Chrome. We were told that Inkling e-books use HTML5 and CSS3 extensively and that not all the HTML5 and CSS3 features that Inkling uses are supported in IE and Firefox.  Okay, but if the Web version is to target institutional subscriptions rather than individual consumers – whom their iPhone/iPad version targets, then why the limited cross-browser functionality? In my opinion, no libraries were going to purchase ebooks that are going to run on web-browsers that half of their library patrons or more do not already have on their computers. Perhaps did Inkling release its Web version in haste and are planning to add cross-browser support as soon as possible? However, when my library inquired about Inkling’s future plan of cross-browser support, we were told that they were planning to add that feature in the future but without any timeline for it. Then, the cross-browser support didn’t seem like a high-priority for them. So if this is the case, what is going on?

Source: http://www.flickr.com/photos/angeliathatsme/6182582677/

As long as I was thinking as a librarian, I could only think that this was simply a bad business model. But let’s think about from Inkling’s perspective.  Is there a possibility that this is not a bad business model but an actually savvy business strategy? I do not doubt that Inkling’s development team can make their platform work with Firefox and IE. From the product they built, Inkling seems to have a capable dev team. I wonder if the company is debating internally whether adding the cross-browser support to their e-book platform would be a worthwhile investment at this point.

(A) First of all, HTML5 is being fast accepted as the new web standard and the W3C has a working group to finalize the specification. So whatever feature Inkling needs and FireFox lacks may well be soon added. And hopefully IE will follow the suit. If the browser support for these features are coming, Inkling may well do better by simply waiting a bit.

(B) The company may be uncertain about how much revenue will be generated from the institutional subscriptions. If the main attraction of their product lies in its availability on iPhone and iPad, then since this feature is not allowed for the users of Inkling’s Web version through the institutional subscription, individual consumers may well remain as their main target customer group. And for that matter, the release of the Web version itself doesn’t necessarily mean that the company is targeting institutional subscriptions as the main customers. The Web version might be a nice companion for the individual users who nevertheless purchase Inkling books mostly to use them on the iOS platform.

(c) But most importantly, Inkling’s real target could be mobile devices, not desktops. Now think about mobile devices – many different kinds of smartphones and tablets. If they are iOS devices, they will have Safari web browser. If not, they are likely to be Android devices. What browser do Android devices have? Chrome. So perhaps what the real market strategy of Inkling is going for the mobile market, not the desktop market. Maybe this is what they are really after.  Now the choice of Sarafi and Chrome seems to make much more sense, doesn’t it? And I have to say that focusing on the mobile is only smart considering that the Mobile Internet is about to surpass the Desktop Internet.  Of course, I have no way of knowing if any of these guesses are true. I am simply testing hypotheses.

So what was the lesson I learned? It was something obvious but nevertheless I have been overlooking for a while. As librarians, we are used to evaluate a resource product based upon how user-friendly it is for library patrons to use and how easy it is to implement in the common library setting (such as IP authentication by EZproxy and institutional log-in). However, these two factors are not necessarily the greatest concern for the vendors. That doesn’t mean that their business model is bad. They just happen to have a different business model that is not quite library-friendly. But vendors don’t exist to be library-friendly as much as libraries exist to purchase vendor products.  And sometimes, thinking about their concerns rather than ours can give us librarians a clue of where the vendors are going with their products and what their responses to our requests are likely to be, which we need to be aware of and to understand as much as we can. It is a good mental exercise for librarians.

 

 

The simplest AJAX: writing your own code (1)

*** This blog post has been originally published in ACRL TechConnect 0n September 12, 2012. ***

It has been 8 months since the Code Year project started. Back in January, I have provided some tips. Now I want to check in to see if how well you have been following along. Falling behind? You are not alone. Luckily there are still 3-4 months left.

Teaching oneself how to code is not easy. One of the many challenges is keeping at it on a regular basis. Both at home and at work, there seems to be always a dozen things higher in priority than code lessons. Another problem is that often we start a learning project by reading a book with some chosen examples. The Code Year project is somewhat better since it provides interactive tutorials. But at the end of many tutorials, you may have experienced the nagging feeling of doubt about whether you can now go out to the real world and make something that works. Have you done any real-time project yet?

If you are like me, the biggest obstacle in starting your own first small coding project is not so much the lack of knowledge as the fantasy that you still have yet more to learn before trying any such real-life-ish project. I call this ‘fantasy’ because there is never such a time when you are in full possession of knowledge before jumping into a project. In most cases, you discover what you need to learn only after you start a project and run into a problem that you need to solve.

So for this blog post, I tried building something very small. During the process, I had to fight constantly with the feeling that I should go back to the Code Year Project and take those comforting lessons in Javascript and JQeury that I didn’t have time to work through yet. But I also knew that I would be so much more motivated to keep learning if I can just see myself making something on my own. I decided to try some very simple AJAX stuff and started by looking at two examples on the Web. Here I will share those examples and my review process that enabled me to write my own bit of code. After looking at these, I was able to use different APIs to get the same result. My explanation below is intentionally detailed for beginners. But if you can understand the examples without my line-by-line explanation, feel free to skip and go directly to the last section where the challenge is. For what would your AJAX skill be useful? There are a lot of useful data in the cloud. Using AJAX, you can dynamically display your library’s photos stored in Flickr in your library’s website or generate a custom bibliography on the fly using the tags in Pinboard or MESH (Medical Subject Heading) and other filters in PubMed. You can mash up data feeds from multiple providers and create something completely new and interesting such as HealthMap, iSpiecies, and Housing Maps.

Warm-up 1: Jason’s Flickr API example

I found this example, “Flickr API – Display Photos (JSON)” quite useful. This example is at Jason Clark’s website. Jason has many cool code examples and working programs under the Code & Files page. You can see the source of the whole HTML page here . But let’s see the JS part below.

<script type="text/javascript">
//run function to parse json response, grab title, link, and media values - place in html tags
function jsonFlickrFeed(fr) {
    var container = document.getElementById("feed");
    var markup = '<h1>' + '<a href="' + fr.link+ '">' + fr.title + '</a>'+ '</h1>';
    for (var i = 0; i < fr.items.length; i++) {
    markup += '<a title="' + fr.items[i].title + '" href="' + fr.items[i].link + '"><img src="' + fr.items[i].media.m + '" alt="' + fr.items[i].title + '"></a>';
}
container.innerHTML = markup;
}
</script>
<script type="text/javascript" src="http://api.flickr.com/services/feeds/photos_public.gne?tags=cil2009&format=json">
</script>

After spending a few minutes looking at the source of the page, you can figure out the following:

  • Line 12 imports data formatted in JSON from Flickr, and the JSON data is wrapped in a JS function called jsonFlickrFeed. You can find these data source urls in API documentation usually. But many API documentations are often hard to decipher. In this case, this MashupGuide page by Raymond Yee was quite helpful.
  • Line 3-8 are defining the jsonFlickrFeed function that processes the JSON data.

You can think of JSON as a JS object or an associative array of them. Can you also figure out what is going on inside the jsonFlickrFeed function? Let’s go through it line by line.

  • Line 4 creates a variable, container, and sets it to the empty div given the id of the “feed.”
  • Line 5 creates another variable, markup, which will include a link and a title of “fr,” which is an arbitrary name that refers to the JSON data thrown inside the jsonFlickrFeed fucntion.
  • Line 6-8 are a for-loop that goes through every object in the items array and extracts its title and link as well as the image source link and title. The loop also adds the resulting HTML string to the markup variable.
  • Line 9 assigns the content of the markup variable as the value of the HTML content of the variable, container. Since the empty div with the “feed” id was assigned to the variable container, now the feed div has the content of var markup as its HTML content.

So these two JS snippets take an empty div like this:

<div id="feed"></div>

Then they dynamically generate the content inside with the source data from Flickr following some minimal presentation specified in the JS itself. Below is the dynamically generated content for the feed div. The result like this.

<div id="feed">
<h1>
<a href="http://www.flickr.com/photos/tags/cil2009/">Recent Uploads tagged cil2009</a>
</h1>
<a href="http://www.flickr.com/photos/matthew_francis/3458100856/" title="Waiting at Vienna metro (cropped)">
<img alt="Waiting at Vienna metro (cropped)" src="http://farm4.staticflickr.com/3608/3458100856_d01b26cf1b_m.jpg">
</a>
<a href="http://www.flickr.com/photos/libraryman/3448484629/" title="Laptop right before CIL2009 session">
<img alt="Laptop right before CIL2009 session" src="http://farm4.staticflickr.com/3389/3448484629_9874f4ab92_m.jpg">
</a>
<a href="http://www.flickr.com/photos/christajoy42/4814625142/" title="Computers in Libraries 2009">
<img alt="Computers in Libraries 2009" src="http://farm5.staticflickr.com/4082/4814625142_f9d9f90118_m.jpg">
</a>
<a href="http://www.flickr.com/photos/librarianinblack/3613111168/" title="David Lee King">
<img alt="David Lee King" src="http://farm4.staticflickr.com/3354/3613111168_02299f2b53_m.jpg">
</a>
<a href="http://www.flickr.com/photos/librarianinblack/3613111084/" title="Aaron Schmidt">
<img alt="Aaron Schmidt" src="http://farm4.staticflickr.com/3331/3613111084_b5ba9e70bd_m.jpg">
</a>
<a href="http://www.flickr.com/photos/librarianinblack/3612296027/" block"="" libraries"="" in="" computers="" title="The Kids on the ">
<img block"="" libraries"="" in="" computers="" alt="The Kids on the " src="http://farm3.staticflickr.com/2426/3612296027_6f4043077d_m.jpg">
</a>
<a href="http://www.flickr.com/photos/pegasuslibrarian/3460426841/" title="Dave and Greg look down at CarpetCon">
<img alt="Dave and Greg look down at CarpetCon" src="http://farm4.staticflickr.com/3576/3460426841_ef2e57ab49_m.jpg">
</a>
<a href="http://www.flickr.com/photos/pegasuslibrarian/3460425549/" title="Jason and Krista at CarpetCon">
<img alt="Jason and Krista at CarpetCon" src="http://farm4.staticflickr.com/3600/3460425549_55443c5ddb_m.jpg">
</a>
<a href="http://www.flickr.com/photos/pegasuslibrarian/3460422979/" title="Lunch with Dave, Laura, and Matt">
<img alt="Lunch with Dave, Laura, and Matt" src="http://farm4.staticflickr.com/3530/3460422979_96c020a440_m.jpg">
</a>
<a href="http://www.flickr.com/photos/jezmynne/3436564507/" title="IMG_0532">
<img alt="IMG_0532" src="http://farm4.staticflickr.com/3556/3436564507_551c7c5c0d_m.jpg">
</a>
<a href="http://www.flickr.com/photos/jezmynne/3436566975/" title="IMG_0529">
<img alt="IMG_0529" src="http://farm4.staticflickr.com/3328/3436566975_c8bfe9b081_m.jpg">
</a>
<a href="http://www.flickr.com/photos/jezmynne/3436556645/" title="IMG_0518">
<img alt="IMG_0518" src="http://farm4.staticflickr.com/3579/3436556645_9b01df7f93_m.jpg">
</a>
<a href="http://www.flickr.com/photos/jezmynne/3436569429/" title="IMG_0530">
<img alt="IMG_0530" src="http://farm4.staticflickr.com/3371/3436569429_92d0797719_m.jpg">
</a>
<a href="http://www.flickr.com/photos/jezmynne/3436558817/" title="IMG_0524">
<img alt="IMG_0524" src="http://farm4.staticflickr.com/3331/3436558817_3ff88a60be_m.jpg">
</a>
<a href="http://www.flickr.com/photos/jezmynne/3437361826/" title="IMG_0521">
<img alt="IMG_0521" src="http://farm4.staticflickr.com/3371/3437361826_29a38e0609_m.jpg">
</a>
<a href="http://www.flickr.com/photos/jezmynne/3437356988/" title="IMG_0516">
<img alt="IMG_0516" src="http://farm4.staticflickr.com/3298/3437356988_5aaa94452c_m.jpg">
</a>
<a href="http://www.flickr.com/photos/jezmynne/3437369906/" title="IMG_0528">
<img alt="IMG_0528" src="http://farm4.staticflickr.com/3315/3437369906_01015ce018_m.jpg">
</a>
<a href="http://www.flickr.com/photos/jezmynne/3436560613/" title="IMG_0526">
<img alt="IMG_0526" src="http://farm4.staticflickr.com/3579/3436560613_98775afc79_m.jpg">
</a>
<a href="http://www.flickr.com/photos/jezmynne/3437359398/" title="IMG_0517">
<img alt="IMG_0517" src="http://farm4.staticflickr.com/3131/3437359398_7e339cf161_m.jpg">
</a>
<a href="http://www.flickr.com/photos/jezmynne/3436535739/" title="IMG_0506">
<img alt="IMG_0506" src="http://farm4.staticflickr.com/3646/3436535739_c164062d6b_m.jpg">
</a>
</div>

Strictly speaking, Flickr is returning data in JSONP rather than JSON here. You will see what JSONP means in a little bit. But for now, don’t worry about that distinction. What is cool is that you can grab the data from a third party like Flickr and then you can remix and represent them in your own page.

Warm-up 2: Doing the same with JQuery using $.getJSON()

Since I had figured out how to display data from Flickr using Javascript (thanks to Jason’s code example), the next I wanted to try was to do the same with JQuery. After some googling, I discovered that there is a convenient JQeury method called $.getJSON(). The official JQuery page on this $.getJSON() method includes not only the explanation about JSONP (which allows you to load the data from the domain other than yours in your browser and manipulate it unlike JSON which will be restricted by the same origin policy) but also the JQuery example of processing the same Flickr JSONP data. This is the example from the JQuery website.

$.getJSON("http://api.flickr.com/services/feeds/photos_public.gne?jsoncallback=?",
  {
    tags: "mount rainier",
    tagmode: "any",
    format: "json"
  },
  function(data) {
    $.each(data.items, function(i,item){
      $("<img/>").attr("src", item.media.m).appendTo("#images");
      if ( i == 3 ) return false;
    });
  });

As you can see in the first line, the data feed urls for JSONP response have a part similar to &jasoncallback=? at the end. The function name can vary and the API documentation of a data provider provides that bit of information. Let’s go through the codes line by line:

  • Line 1-6 requests and takes in the data feed from the speicified URL in JSONP format.
  • Once the data is received and ready, the script invokes the anonymous function from line 7-11. This function makes use of the JQuery method $.each().
  • For each of data.items, the anonymous function applies another anonymous function from line 9-10.
  • Line 9 creates an image tag – $(“<img/>”), attaches each item’s media.m element as the source attribute to the image tag – .attr(“src”, item.media.m), and lastly appends the resulting string to the empty div with the id of “images” – .appendTo(“#images”).
  • Line 10 makes sure that no more than 4 items in data.items is processed.

You can see the entire HTML page codes in the JQuery website’s $.getJSON() page.

Your Turn: Try out an API other than Flickr

So far we have looked through two examples. Not too bad, right? To keep the post at a reasonable length, I will get to the little bit of code that I wrote in the next post. This means that you can try the same and we can compare the result next time. Now here is the challenge. Both examples we saw used the Flickr API. Could you write code for a different API provider that does the same thing? Remember that you have to pick a data provider that offers feeds in JSONP if you want to avoid dealing with the same origin policy.

Here are a few providers you might want to check out. They all offer their data feeds in JSONP.

First, find out what data URLs you can use to get JSONP responses. Then write several lines of codes in JS and JQuery to process and display the data in the way you like in your own webpage. You may end up with some googling and research while you are at it.

Here are a few tips that will help you along the way:

  • Verify the data feed URL to see if you are getting the right JSONP responses. Just type the source url into the browser window and see if you get something like this.
  • Get the Firebug for debugging if you don’t already have it.
  • Use the Firebug’s NET panel to see if you are receiving the data OK.
  • Use the Console panel for debugging. The part of data that you want to pick up may be in several levels deep. So it is useful to know if you are getting the right item first before trying to manipulate it.

Happy Coding! See the following screenshots for the Firebug NET panel and Console panel. (Click the images to see the bigger and clearer version.) Don’t forget to share your completed project in the comments section as well as any questions, comments, advice, suggestions!

Net panel in Firebug

 

 

Console panel in Firebug

The Mobile Web and the Mobile Websites of Libraries

I have given a talk at the 2012 Amigos Online Conference – Access by Touch: Delivering Library Services Through Mobile Technologies last Friday. Thanks to all who attended my session!

I won’t be really write any post on my blog on this until I come back from the ALA Annual Conference. But I wanted to share the slides for now. And for those who rather prefer words than slides, here is the super-short summary of my talk.

  • We are moving into the era of the Mobile Web which is now a competitor to the Desktop Web.
    (If you haven’t heard about the Mobile Web, it’s a good time for you to do now!)
  • The user behavior on the Mobile Web is significantly different from that in a few years ago already.
  • The new mantra for the Mobile Web is “Don’t Dumb Things Down on Mobile.”
  • In the last few years, library mobile Web sites have been changing slowly but with some definite trend. The trend is more and more support for the use of library resources and account transactions on the mobile itself as one of the primary tasks on a library’s mobile Website.
  • So my recommendation is:
    • If you library doesn’t have a mobile site yet, build one.
      (In the poll I did, 70% of my audience reported that their libraries do not yet have a mobile website.)
    • Ensure discoverability
      : If your library has a mobile site, please enable redirect (either auto or with a choice) to the mobile version so that a user can discover your mobile site in the most perfect context, i.e when they are getting to your library website on their mobile.
    • Then think about updating your mobile site in keeping with the recent trend of the Mobile Web.

I am also chairing the Mobile Computing IG meeting at the 2012 ALA Annual Conference.
So if you are interested and will be at Anaheim, come by to the meeting!

Slides: The Mobile Web and the Mobile Websites of Libraries: How They Changed for the Last Few Years

View more PowerPoint from Bohyun Kim