As part of my long-term plan for Eviduction, I wanted an easy, automated way to extract notes from my kindle into a centralized management system. After a few days, I had a fully functional program in DaleyKlippings.
If you are looking for more information on the DaleyKippings software (including Downloads and Help), please go to the homepage of the software.
For information on the project’s genesis, read on:
First, I had to find a piece of note management software that met my needs. After some research, I decided to give Evernote(R) a try. The program shares notes across platforms and permits tagging of note and sorting of tags (although a full tag hierarchy isn’t yet supported). Most importantly, the program offers a free tier. It doesn’t currently integrate with Mendeley(R) (my preferred citation manager) or an authoring program like Microsoft Word(R) or Scrivener. However, open source alternatives were too rudimentary to be of much use.
Next, I needed to figure out how to extract notes from the Kindle and get them into Evernote. My research identified several options, but most of them didn’t address my needs. A week later, I had developed DaleyKlippings.
The project started when I found a public domain solution called Klippings. This program was designed with my goal in mind, extracting notes from the Kindle and exporting them into any format you desired. The program was extremely attractive because the export patterns were highly customizable. After a little work, I developed an Export pattern that was compatible with Evernote’s ENEX files.
Unfortunately, the original Klippings software had several (some major) issues:
- The program assumed that Notes were listed after their associated Highlight. This was not true of my Kindle Touch.
- Matching between Notes and Highlights were buggy. The match was using only date/time and title. This created frequent mismatches.
- The program didn’t distinguish between Title and Author… or Location and Page. Evernote explicitly supports an author field and there was no reason to lose Location and Page data that could be important to proper citation in the future.
- Evernote’s Tags were limited to 100 characters and didn’t support commas. Since commas were frequently used in the book name, imports often failed.
Fortunately, it was written in Python and I had recently learned the language for my eSongbook project. Over the next few days, I made a series of major improvements including:
- Advanced matching between notes and highlights:
- The matching algorithm now uses locations and location ranges instead of dates when determining matches. This should significantly improve the accuracy of the matching process.
- When attaching a note, the algorithm automatically checks the highlight before and after the note. This expands support to Kindle devices (like the Touch) that list the notes before the highlights.
- Now supports Author, Page, Note, and Highlight fields
- All of these fields can be used as wildcards when creating export templates
- Generally speaking, users should continue to use “Text” when creating export patterns because “Note” and “Highlight” will not always have data.
- Added “formatting prefixes” – Formatting prefixes make it easier to adjust data based on your output needs. Current formats include:
- XmlSafe – Replaces <, >, and & with their HTML equivalents
- EvernoteTag – Enforces Evernote’s tag requirements by removing all commas (replaced with ‘_’) and limiting length to 100 characters
- Formatting prefixes can be added to any tag to change the output. For example, XmlSafeTitle or XmlSafeText.
- Fully customized Attached Notes
- Notes no longer use the “highlight delimeter” logic. Instead, each export pattern has an extra input box. This box accepts a pattern that will be applied to Attached Notes.
- This box uses the same pattern logic as other export windows. It accepts all prefixes and any field EXCEPT “Text”.
- Enhanced Text field
- Despite changes to Attached Notes and user interface, the Text field will continue to dynamically choose between “Bookmarks” (empty), “Notes”, “Highlights”, and “Attached Notes” when filling in the field.
- The Text field accepts all Prefixes with one caveat. Because Attached Notes can be fully configured through the Export pattern, prefixes are not applied to Attached Notes. These prefixes are applied to individual Notes and Highlights.
While I could have kept the project to myself, I wanted to share it with the community. As a result of the licensing agreement of several of the underlying libraries, I have currently released the project under a GPL license.