@brohrer I myself on Saturday telling me that there is a beer in the fridge while the potatoes are boiling.

I guess your question was discussed already a few thousand years ago. I wonder what it means that we still wonder.

Maybe nobody wants to know.

@arroz And, yeah, why there are so many companies who send this PDFs. God knows. I worked in the automotive industry until 2015 and they still faxed orders.. And it's not Australia only, e.g. just recently we "OCRed" a big Canadian company's invoices.

@arroz I don't have the exact numbers of "traditional" OCR but it will be around 90% as well. And, yes, you are right, the issue is not to get the letters right, it's to make it structured information. With OCR it needs templating which tells the OCR where to find an address, what to do with multiple lines and pages etc. Every new format requires that work again.

LLMs are "smarter" in that regard.

Fun fact rookie error: Sending a T&C page to a LLM. It chews on it forever..

@arroz Also, if there still is an error in one invoice and purchase order, it is usually not catastrophic. You get 250 screws instead of 25.. that happened even before we had computers. It's annoying but.. well, magic doesn't happen, sh** does

Given that we work on behalf of customers, we need to have an acceptably low error rate, of course.

@arroz In this case there are invoices and purchase orders coming as PDF, unstructured data.

Currently there is OCR software and manual data entry. Both make mistakes, so there is always "double keying". If the result is the same, it is considered right. Otherwise it goes to review.

Now there are 2 LLMs who do the "keying" job. Both get it ça. 90% right.

A difference to compilers: two compilers do not create the same machine code, so one cannot compare two results and decide that's right.

@arroz In this case, the LLMs are replacing a boring job to a certain extend.

I wouldn't trust a "90% right" machine a job where people's lives can depend on it, though.

Also, there are traditional OCR based solutions used before and concurrently. In this project the jury is still out. Not certain which is more efficient. The obstacles and issues are bigger than expected. Not all smooth sailing.

@arroz It is funny, even people who work for months on a LLM project are surprised that the LLM does not give consistently the same result.

Which can be ok, in some cases. In the one Isee right now, replacing boring data entry, the LLM gets a result 90% right, and if a second one independently gets the same result, the result is considered confirmed - it is in fact very unlikely that two models get the same thing wrong.

Leaves 20% for review, and the LLMs are faster than humans.