> "...you are better off making your own models to do this"

I'm doubtful you meant what you wrote here. Using a readymade UI or API to perform an effectively magical task (for most of us) is an entirely different paradigm to "just train your own model."

In reality, for us non-ML model training mortals, we're actually probably better off hiring a human to do basic data entry.

So is 4o. Problem isn't the absence of model, it's inconsistency.

I don’t think any prompting skill guarantees the absence of hallucination. And if hallucination is possible, you will usually need to worry about it

As soon as you have something else than a paragraph in a single column layout, you will get hallucinations, random stuff, cut off etc even if you say which pages to look at, LLM will just do what ever.

Can you give some examples of prompts that you use that will tackle hallucinations?

> I think it offers an optimal balance of affordability & reliability.

It is hard to trust "you" when ChatGPT wrote that text. You never know which part of the answer is genuine and which part was made up by ChatGPT.

To actually answer that question: Pricing varies quite a bit depending on what exactly you want to do with a document.

Text detection generally costs $1.5 per 1k pages:

https://cloud.google.com/vision/pricing

https://aws.amazon.com/textract/pricing/

https://azure.microsoft.com/en-us/pricing/details/ai-documen...

Was looking for a self-hosted solution as I have quite on/off needs, but I'll give it a whirl as it looks quite promising.

+1! Most LLMs can already output Mathpix markdown. I prompt it to do so and it gives the code and this use a rendering library to show the scalable and selectable equations. No wonder facebook nougat also uses it. Good stuff!

>then just start making up numbers...

Absolutely! It's a fucking criminal in that regard. But that's why everything is done with hard python code and the results are tested multiple times. As an assistant, gpt can be fabulous, but the user must run the necessary scripts on their own and be ever ready for a knife in the back at any moment.

Edit: below is an example of what it generated after a lot of debugging and hassle:

  import re

def clean_and_structure_data(text): """Cleans and structures the extracted text data.""" # Regular expression pattern to match the lottery data pattern = r'(\d{2}/\d{2}/\d{2})\s+(E|M)\s+(\d{1})\s-\s(\d{1})\s-\s(\d{1})\s-\s(\d{1})(?:\s+FB\s+(\d))?' matches = re.findall(pattern, text)

    structured_data = []
    for match in matches:
        date, draw_type, n1, n2, n3, n4, fireball = match
        # Format the date to include the full year
        date = datetime.strptime(date, '%m/%d/%y').strftime('%m/%d/%Y')
        # Concatenate the numbers, ensuring leading zeros are preserved, and enclose in quotes
        numbers = f'"{n1}{n2}{n3}{n4}"'
        structured_data.append({
            'Date': date,
            'Draw': draw_type,
            'Numbers': numbers,
            'Fireball': fireball or ''  # Use empty string if Fireball is None
        })
    return structured_data

    with open(output_path, 'w', newline='') as csvfile:
        fieldnames = ['Date', 'Draw', 'Numbers', 'Fireball']
        writer = csv.DictWriter(csvfile, fieldnames=fieldnames)
        writer.writeheader()
        for row in sorted_data:
            writer.writerow(row)

    try:
        with open(txt_path, 'r') as file:
            text = file.read()
       
        cleaned_data = clean_and_structure_data(text)
        save_to_csv(cleaned_data, output_csv_path)
        print(f"Data successfully extracted and saved to {output_csv_path}")
    except Exception as e:
        print(f"An error occurred: {e}")

I am honestly not certain why they obstruct easy access to the number history. It's obviously accessible, but only through manually parsing the PDF. Their prior embedded search function, approximately two years ago, would return all permutations of the queried number from day 1 to present. They modified it to exclude results more than two years old. The PDF contains the entire data set, but isn't searchable. Why? Dunno. But I'm cynical

I've also compiled a list of all numbers that have never occurred, count of each occurrence and a lot more. My anomaly analytics have included everything, as an ignoramous, I can throw at it; chi squared; isolated forest; time series; and a lot of stuff I don't properly understand. Most anomalies found have been, if narrowly, within expected randomness, but I intend to fortify my proddings eventually. Although I'm actually confident I'm barking up the wrong tree, the data obfuscation is objectively dubious, for whatever the reason.

Thanks for asking...

1) I'm a rebel

2) I am irritated by deliberate obfuscations of public data, especially by a source that I suspect is corrupt. Although my extensive analysis has not yet revealed any significant pattern anomalies in their numbers.

3) It's kind of my re-intro into python, which I never made significant progress in but always wanted to.

4) It's literally the real history of all winning numbers since inception. Individuals may have various reasons for accessing this data, but I've been using it to test for manipulation. I presume for most folks it would be curiosity, or gambler's fallacy type stuff. Regardless, it shouldn't be obfuscated.