eReader: A Library in the Cloud

You have an eReader. Or maybe just some eBooks from all those free classics sites. Or you have some eARCs from NetGalley.

If you're like me, you've probably downloaded all your books and stored them on your computer in some kind of folder organization. Maybe you're using folders to organize your books by their source (NetGalley, B&N, Google Books, etc.). Or maybe you're using folders to organize your books by genre, shelf, or series. When you want to read your books, you plug your eReader or your smart phone to the computer and move everything over by hand.

There's nothing wrong with your system - it works pretty well. You've got a place for everything and everything in its place. But it is a little irksome that you either have to keep your entire library loaded to your reader at all times (which can become more and more unmanageable as it grows) or you have to choose in advance which books you want to take on the go with you. Plus, as much as you like your folder system, it can be a little unwieldy at times... and it would also be nice to have your library backed up automatically to a storage facility in case anything ever happens to your home computer.

So I'm going to tell you how I organize my library in the cloud with a few simple programs - all of them (mostly) free. This system lets me organize my books, backs them up automatically to online storage, and allows me to access them at any time from a 3G smart phone or a WiFi tablet (including a rooted Nook Color). I also want to give credit that I originally learned this method from the Dear Author blog, and have merely tweaked it a little with my own screenshots and system.

Step 1: Using Dropbox for cloud access and storage.

The first thing you want to do is create a Dropbox account. You can go to their website directly or use this referral link. Creating a Dropbox account is free - there's an optional yearly subscription you can choose to pay for more storage, but you don't ever have to subscribe if you don't want to.

There are a lot of online storage facilities, and honestly Dropbox is the only one that I've found will work with this eBook organization-and-remote-access method. The reason for this is that we're going to load an HTML page that contains your library catalog to the online storage facility, and you have to be able to access that HTML page as if it were any other HTML page. Dropbox seems to be the only online storage program that lets you do that - other storage programs like SugarSync will only let you download the HTML page as just another file, and then it doesn't work.

The "downside" to this Dropbox method is that while Dropbox gives you two folders (a "private" folder where you can control who can access it via a username/password combination and a "public" folder where you can control who can access it only by keeping the link to your public folder secret and safe), you have to use the "public" folder to store your eBook library - the "private" folder mucks with the relative links in your HTML library file. And the issue with the "public" folder is that once someone has the link to something in that public folder, they can access that link at any time unless you take down the item or otherwise change the link.

The "upside" to all this is that Dropbox doesn't submit its links to any search sites or web crawlers. Your links will therefore stay private until you share them with someone else or post them on a webpage. Therefore, don't do that. The only other way anyone can ever get to your Dropbox material is by guessing the link, so the easiest way to fix this is to nest everything in your Dropbox folder in a "gatekeeper" folder that contains a word or phrase only known to you, followed by generated GUID. Now the chances of anyone guessing your folder path online is pretty much zero.


Install Dropbox to your home computer wherever you would like your Calibre library to live. Set up your "gatekeeper" folder, and drop it into the "Public" Dropbox folder; then nest another folder inside your gatekeeper folder and call it "Calibre" or "Library" or something similarly meaningful to you.

Step 2: Using Calibre for local organization.

The next step in all this (and the first step, really, if you just want local organization without the cloud access) is to download and install Calibre. Calibre is the best program I know for organizing your eBook library locally on your computer, and you'll also need it for an independently developed plug-in that will use the Calibre data to create an HTML page for accessing all your books remotely.

In Calibre, you can use tags to organize your books - I use tags to track genre information, as well as where I got the eBook. You can change the metadata on your epub books, such as the cover picture, which is a big deal for me since most of my B&N eBooks come with plain, uninteresting covers. Changing the cover in Calibre is easy - edit the metadata, upload a different cover pic, run a quick "conversion" (from epub-to-epub) to make the metadata stick, and you're finished.

Here's a picture of my own Calibre library:


In order to move your Calibre library to the Dropbox directory you set up, click the "books" icon at the top of the program and a prompt will come up to let you move your library to a new location.


Set the location to your Dropbox directory, via the format "Dropbox\Public\<gatekeeper folder>\<calibre library folder>".

Step 3: Using Calibre2opds to see your library at a glance.

Once you have Calibre installed and set up in the appropriate directory, the next step is to download and install Dave Pierron's Calibre2opds program (the download links are on the right). What this program does is it interfaces with the Calibre library in order to create a "library at a glance" HTML catalog (or XML catalog - we'll get to that in Step 4) that contains relative links to all your Calibre eBooks. What's valuable about this is that you can browse through your entire eBook library by cover, by author, by tags, by rating, or by recent addition, and you can download the books directly to your reader remotely, as long as you have the private link to this HTML index.

Once you've got Calibre2opds installed, boot it up and fill in the following data:


Once you've entered your library paths and whatever name you want the catalog folder to have, hit "Generate Catalogs".The program will then create an "index.html" file in your created catalog folder that will contain all the data from your Calibre library as well as relative paths to the books contained there in.




You now have two index files in your Dropbox catalog folder - an "index.html" and an "index.xml". Depending on which reader you use (see Step 4), one of these files will be used to access your catalog remotely. In order to get the web link to that file, navigate to the index file, right-click the file, and select Dropbox --> Copy Public Link. Paste that link into a text file somewhere so that you can see what the link is.

Step 4: Using a reader application to access your catalog from your device.

Now go get your tablet or smart phone. If you're using an Android device (including the rooted Nook Color), you'll want to download and use either Aldiko or Moon+. (I use both - I like Aldiko best for overall reading, but currently the Moon+ has far better highlighting, sharing, and note-taking options.) If you're using an Apple device, then you'll want to download and install Stanza.

If you're using Aldiko: Set up your Calibre2opds catalog with the "Aldiko" compatibility option, and copy the public link to the index.html file in your Dropbox catalog. Navigate to the Aldiko "Home" screen. Select "My Catalogs". Press the "+" button to add a catalog. Type in the public link to your index.html file and save the catalog link. Now when you use Aldiko, you can click on "My Catalogs", select your saved Dropbox catalog, and then interface with your online library using the pages that Calibre2opds created for you. You can download books directly to your device over a 3G/WiFi connection, no matter where you are.

Aldiko Home Screen
Aldiko Shelf View

If you're using Moon+: Set up your Calibre2opds catalog with the "Stanza" compatibility option, and copy the public link to the index.xml file in your Dropbox catalog. Navigate to the Moon+ "Net Library" screen. Select "+ Add New Catalog". Type in the public link to your index.xml file and save the catalog link. Now when you use Moon+, you can click on "Net Library", select your saved Dropbox catalog, and then interface with your online library using the pages that Calibre2opds created for you. You can download books directly to your device over a 3G/WiFi connection, no matter where you are.

Stanza Home Screen
Stanza Highlighting Options

If you're using Stanza: I don't have an Apple device, but you can see how to set up Stanza from the original article.

Step 5: Using profiles and batch scripts to work smarter, not harder.

Calibre2opds offers the ability to save a profile with the given program options - you can access these by using the "Profiles" menu at the top of the program. I find it useful to make three profiles:

  1. One named "AldikoBrowseByCover" that has the "Aldiko" compatibility option, the "Catalog Generation Options --> Browse By Cover" mode checked, and is saved in a "_catalog-covers" directory. 
  2. One named "AldikoBrowseByLetter" that has the "Aldiko" compatibility option, the "Catalog Generation Options --> Browse By Cover" mode unchecked, and is saved in a "_catalog-letters" directory. 
  3.  One named "StanzaBrowseByLetter" that has the "Stanza" compatibility option, and is saved in a "_catalog-stanza" directory.
(More information on the Calibre2opds preferences can be found here.)

Since Dave Pierron has set up a great command line interface for his program and since I don't want to manually re-generate the catalogs every time I add a book to Calibre, I decided to set up a batch script that generates the catalogs, and then use Windows Task Scheduler to schedule the computer to run the batch script automatically once a day. You can do this too: create a text file and re-name it something like "Generate_Calibre2opds.bat". Right-click the file and select "Edit" and paste in the following batch script:

:: This is a batch script meant to be run from Windows Task Scheduler.
:: Note that the 'start' command is used below to prevent the thread from exiting
:: immediately after the first run call.
:: Note also that the 'run.cmd' file was edited to include an 'exit' command so that
:: the three threads below don't stay open afterwards.

:: This batch script changes the directory 
:: in order to run the Calibre2opds command line script.
cd "C:\Program Files (x86)\Calibre2Opds"

:: Run the command line with the saved Aldiko covers profile.
start run.cmd AldikoBrowseByCover

:: Run the command line with the saved Aldiko letters profile.
start run.cmd AldikoBrowseByLetter

:: Run the command line with the saved Stanza letters profile.
start run.cmd StanzaBrowseByLetter

You would, of course, change the red profile names to whatever you had named your profiles. (More information on the Calibre2opds command line generation can be found here.) From there, it's a relatively simple task to set up the Windows Task Scheduler to run the bat file daily. (Feel free to email me for more details, if you need help.)

22 comments:

Jeania Kimbrough said...

This is interesting. Thanks for sharing. I have Calibre but haven't been using it this way.

MrsJoseph said...

Hey Anamardoll! 

Question, does this work with Sugarsync?  I already have that one.

Ana Mardoll said...

Unfortunately, no, SugarSync does not allow the direct access to the HTML file as a webpage that Aldiko/Stanza/Moon+/etc. requires in order to browse the catalog.

You can still put your books into SugarSync and download them to an android device with the SugarSync app, but you don't get the direct catalog yumminess that Calibre2opds offers. :(

MrsJoseph said...

dang it.  Guess I have to switch everything over to dropbox. 

Toodles said...

I am new to the Calibre/reader app game, but thank you so much for the easy to understand procedures.  As I am a complete newbie, I've got a gee whiz question: I have an HP touchpad (webOS) with a Kindle app.  I was tracking with you all the way until step 4.  Do I sideload my xml to the kindle app "catalog" or do I have to manually sideload each .mobi file?  Also, as I am using firefox, I didn't get an "html" index, just the "xml".

Please, do you have any suggestions?  Thanks!

Toodles said...

Quick follow up:  I do have an .html file, but Windows 7 shows it is a "firefox document".  Also, I went back and regenerated the catalog with the OPDS device option since I am using the Kindle app. 

I think I've got it.  Thank you again for the great instructions!

Ana Mardoll said...

Toodles, if you set the Calibre2opds to the "Stanza" option, it will only give you the XML file -- try setting it to "Aldiko" to get the HTML file. Once you have the HTML on Dropbox, you should be able to navigate to its Dropbox "public link" (which you can get by right-clicking the HTML file on your desktop computer and clicking the Dropbox --> Copy Public Link option) via a web interface.

Most Apple and Android users access the catalog via a reading app that supports catalogs, but I'm not sure that WebOS has any readers that support catalog interfacing just yet.

Toodles said...

Ana, thank you so much.  I did find the HTML file.  At the end of the day (literally), I figured out exactly what you said: WebOS does not have a reader that supports catalog interfacing yet.  Aaaaack!  However, if they ever do (and I know it is a big "if"), I will know what to do.  Thank you again.

Ana Mardoll said...

Something like a billion people bought HP Touchpads this week, what with the clearance sale, so you are in good company. Aldiko, Stanza, and Moon+ developers could be contacted and asked nicely -- failing that, there's a possibility that someone may figure out how to load CM7 on the HP Touchpad. I nearly got one on spec just for that reason. Good luck!!

Alyk Kyla said...

Thanks for the great article. However, I'm having a bit of a problem getting the catalogs to download to Aldiko. I am able to acess the main catalog page, but when i go and click on a book, either the link never connects (requiring me to force close aldiko and restart) or it gives me an error. Any advice?

Also, I've been trying to use the same method with Moon+ and using the catalog in the STANZA format. Moon+ for some reason cannot find the catalog online, although Aldiko has no problems finding it. Any advice on this?

Thanks again for the awesome instruction. Previously, I would just place my books into dropbox, and also save the locally on my harddrive using Calibre. And, since my nook is now rooted and cannot interact with Calibre directly, this is a much better option.

Ana Mardoll said...

One thing I should have clarified: If you are running off an Aldiko catalog AND a Stanza catalog, they need to be stored in different directories -- the outputs are very different because the apps are different. I'm wondering if maybe the two catalogs are overwriting each other in your case and causing the problems you're seeing?

So what you should end up with is something like:

folder1/index.html <-- Aldiko points to this
folder2/index.xml <-- Moon+ points to this

This will, I hope, solve your problem. :)

Alyk Kyla said...

Ahh okay so my problem with Moon+ was that I wasn't using the .xml file. But, unfortunately, after getting to the dropbox public link in both apps, after click the book, in Moon+'s case it just doesn't download, and in Aldiko's case its just a blank page. I'm not sure if I did something wrong since it was working before with Aldiko...any suggestions?

Ana Mardoll said...

And you've re-generated the catalogs in two different directories?

Alyk Kyla said...

Just to be clear, the problem is that, after generating the catalog with the calibre2opds program ( I redid the entire procedure again) when I click on the index.html file, the books are present, however when you click on their respective picture, a blank page comes up, as opposed to the brief summary and download link that you have shown in the third picture under the calibre2opds instructions.

Ana Mardoll said...

Hmm. Well, couple of thoughts.

1. It does take awhile for all the new files to upload to Dropbox. I usually generate the catalog last thing at night and then everything is uploaded by morning. So it's possible that the book page that the html file is linking to isn't uploaded yet, so Aldiko is trying to load a non-existent link. Is your Dropbox folder on your computer fully synced yet? (There will be a green check mark.)

2. If the Dropbox sync is complete, open the html file directly from your desktop computer (i.e., navigate to the folder and double-click on the html file). Click on the book in question. Does that take you to the correct page? If you're getting a blank page from your computer, too, then that means the catalog isn't generating correctly and we'll have to look at how Calibre2opds is set up on your machine.

Alyk Kyla said...

Yes, the dropbox folder is completely synced , and on the computer it is generating a blank page, I guess it Calibre2opds?

Ana Mardoll said...

I'm so sorry you're having this problem, but I'm actually glad to hear it's broken on the computer, too. If you don't mind emailing me (anamardoll@gmail.com), I can send you screenshots of how I have the options set up, and we can see if yours is set up the same way? :)

Terry said...

I've never used dropbox, and of course nothing else you talk about here, so I have a fundamental question: does this process require copies of all of your books to upload to dropbox, or only the index file (with the hyperlinks causing the upload and download of the individual books)?

Ana Mardoll said...

Copies of your books upload to Dropbox, yes. That is why it's so important not to share the link with anyone else and to make it something not-guessable, because you don't want to share your books out with the world. The idea is that only you can access these.

Anonymous said...

Hey there! Quick question that's completely off topic. Do you know how to make your site mobile friendly? My blog looks weird when viewing from my apple iphone. I'm trying to find a theme or
plugin that might be able to resolve this problem.
If you have any suggestions, please share. Many thanks!
Also visit my page ; how to make an app

Anonymous said...

I loved as much as you'll receive carried out right here. The sketch is attractive, your authored subject matter stylish. nonetheless, you command get bought an impatience over that you wish be delivering the following. unwell unquestionably come more formerly again since exactly the same nearly very often inside case you shield this increase.
Also visit my weblog : how to prevent acne from shaving

Anonymous said...

I was more than happy to uncover this site. I want to to thank
you for ones time due to this fantastic read!! I definitely savored every part of it and i also have you book-marked to look at new stuff on
your site.
My web site: how to get rid of pimples

Post a Comment