Using OCR (optical character recognition) to extract image contents to text

By default, images that you upload into an Amplenote note or jot -- whether in the body of the note or in a Rich Footnote -- get indexed via OCR so it can be found in search. This page will detail how Amplenote OCR works, including its current limitations.

linkImage OCR features

OCR is initiated any time you insert an image into a note in Amplenote and you're online. When the image is uploaded to the Amplenote backend, the image text is extracted and merged back into your image in the note. From there, the text will get found in search, or can be inserted into the note. It works something like this:

Using OCR icon to paste an image text into note

You don't need to paste the image text into the note to have it indexed by search -- that happens automatically. But sometimes it's useful to get the text of an image if you want to edit said text.

linkOCR extraction use cases

Some cases where we've seen OCR text pasting come in handy:

link🗒️ Written meeting notes

When we were choosing our OCR engine backend, we emphasized picking a solution that was robust enough to work with handwritten text. In our testing, Amplenote's OCR is reliably able to transform a pad of jotted notes into a format that you can work with inside a note.

link🥒 Meal recipes

A lot of people use Amplenote to capture recipes online. The OCR pasting allows you to extract a single section (like the ingredients) and print that out, or reference it in another note.

link💻 Editing code

The OCR engine is very accurate in translating block image text into raw, editable code. The most common limitation is differences in punctuation.

link🎫 Business cards

Snap a picture of a business card and gets its contents editable in Amplenote so you can create notes with the contact's name

link✍️ Handwritten journaling or "doodling"

As with "written meeting notes," Amplenote's OCR is great if you want to index the text of your past journals to be searchable for your later amusement.

linkConditions for successful OCR processing

In theory, every image uploaded in a note is supposed to have its content indexed into search. However, as of January 2021, there are a handful of known circumstances when OCR may not be successfully completed.

linkSystem limitations

These ones are unlikely to change in the near feature:

OCR requires connecting to an external API, so it only works when online. However, if you upload an image in a note and come online later, that note should have its OCR indexed once the note is visited while online.

For images of 10,000 or more pixels, the resolution of the document sent to the OCR provider tends to be so low that words get clipped. OCR works best for images that are under 5,000 pixels tall.

linkSystem-specific limitations & debugging

In some browser configurations, we have received reports that OCR doesn't index content pasted in. Since any image that enters the note (uploaded via icon, pasted via buffer, etc) is put in a queue to have its OCR processed and then merged back into the note (assuming internet access is available), it's likely that some issues may exist.

If you are online and you have uploaded an image that after ~30 seconds hasn't received the OCR icon, let us know. 🙂 To be able to use your report to bring about improvement, please email the following details:

The browser and platform being used

The image that you upload that doesn't get indexed

A detailed description for what you do in the 20-30 seconds after inserting the image to get the issue to reproduce the OCR failure

A video of the process is optional and very helpful if available. macOS allows recording videos using Cmd-Ctrl-Shift-5.

We will investigate at our earliest convenience to continue to reduce the circumstances under which OCR fails to be expediently applied. In the meantime, your chances of having OCR inserted successfully is to minimize changes to the note in the immediate aftermath of inserting the image.