Extract table data with Docling
Still looking for a way to extract table data from image
In the article two posts ago I used mPLUG-DocOwl2 to extract text from images, and in the last one I used Phi-3 Vision.
I'm currently looking for a less costly way to extract table data from images.
This time I tried out Docling.
What is the Docling?
Docling is a tool that allows you to parse different document formats and export them to your desired format easily and quickly. It can read PDF, DOCX, PPTX, XLSX, images, HTML, AsciiDoc and Markdown now, then output the loaded contents as HTML, Markdown, or JSON.
According to the "Architecture" page below, for each type of file, Backend, PipelineOptions, and Pipeline are defined, which convert the file into a Docling document. Then, the file is converted into the specified output file format.
Docling introduces a unified document representation format called DockingDocument, which is defined as a pydantic datatype capable of expressing several common features of documents, such as text, image, table, sections, header, footer, layout information and so on. For the details, please check this page.
Extract table data from image
This time, I convert table data from images and PDFs into markdown with docling command. At first, we can install it with pip.
pip install docling
Next, I convert table_eng.jpg to markdown.
docling --to md ./table_eng.jpg
Then, it generated table_eng.md.
| Name | Address |
|------------------------------|-------------------------------------------------------------|
| I3th Street | 47 W 13th St, New York, NY 10011, USA |
| [20 Cooper Square | 20 Cooper Square New York NY 10003 USA |
| [2nd Street Dorm | 1E 2nd St New York NY 10003, USA |
| 3rd North | 75 3rd Ave New York NY 10003 USA |
| 6 Metrotech Center | Metrotech Center Brooklyn NY 11201 USA |
| 721 Broadway | |
| 7th Street | 140 E 7th St, New York; NY 10003, USA |
| B838 Broadway | B38 Broadway New York NY 10003 USA |
| Abu Dhabi House | 19 Washington Square N New York NY 10011 USA |
| Affinia Manhattan Hotel | 371 7th Ave, New York, NY 10001 |
| Alumni Hall | |
| Bobst Library | 70 Washington Square South New York NY 10012, United States |
| (Brittany Hall | 55 East lOth Street New York NY 10003 United States |
| Broome | |
| Brown Blde | [29 Washington PI New York, NY 10003, USA |
| [Cantor Film | 36 East Bth Street, New York NY 10003 United States |
| | 25 Union Square W New York NY 10003 USA |
| [Clark Residence Hall (Poly) | 55 Clark St Brooklyn NY 11201 USA |
| Coles Center | 181 Mercer Street New York NY 10012 United States |
| College of Dentistry | 1345 E 24th St, New York, NY 10010 USA |
| Computer Center | [242 Greene St; New York, NY 10003 USA |
| Coral Tower | East 14th Street, New York, NY 10003, United States |
| D'Agostino | 110 W 3rd St, New York NY 10012 USA |
| Ehrenkranz | 13 Washington Square 5 New York NY 10012 USA |
| Faculty of Arts | 5 Washington Square 5, New York NY 10012 USA |
| Founders Hall | 120 E 1Zth St, New York NY 10003 USA |
| Furman Hall | 1249 Sullivan St, New York; NY 10012, USA |
| | Washington Mews New York NY 10003 USA |
| Goddard Hall | 80 Washington Square E New York NY 10003 USA |
| Gramercy Green | (310 3rd Ave, New York, NY 10010, USA |
| (Greenwich Hotel | (636 Greenwich St, New York, NY 10014, USA |
| Health Services | 726 Broadway New York NY 10003USA |
| HKMC | 44 W Ath St, New York NY 10012, USA |
| Kaufman [Kevorkian | 44 W 4th St, New York, NY 10012, USA |
| | 50 Washington Square 5 New York NY 10012 USA |
There are cases where the beginning of the output string is (, [, or 1. This is because the lines in the image are mistakenly recognized as characters. This issue also occurred when I tried img2table in the past.
Looking at the warning output when executing this command, it appears that the image is being read using EasyOCR.
WARNING:easyocr.easyocr:Downloading detection model, please wait. This may take several minutes depending upon your network connection.
WARNING:easyocr.easyocr:Downloading recognition model, please wait. This may take several minutes depending upon your network connection.
The supported OCRs are listed in the FAQ. At this time, in addition to EasyOCR, Tesseract, RapidOCR, and Mac OCR seem to be available.
From what I have tried so far, mPLUG-DocOwl2 and Phi-3 Vision have shown to be less susceptible to this type of noise than these OCRs. Therefore, when it comes to extracting table data from images, Docling may be less accurate than them. Of course, it depends on the image, so I recommend you try it yourself.
Extract table data from PDF
I also tried extracting table data from a PDF file.
!docling --to md ./Saferidestoptable.pdf
The results are as follows:
| | Name |
|------------------------------------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| 13th Street | Address 47 W 13th St, New York, NY 10011, USA |
| 20 Cooper Square 2nd Street Dorm | 20 Cooper Square, New York, NY 10003, USA 1 E 2nd St, New York, NY 10003, USA 75 3rd Ave, New York, NY 10003, USA |
| 3rd North | |
| | Metrotech Center, Brooklyn, NY 11201, USA |
| 6 Metrotech Center 721 Broadway | |
| | 721 Broadway, New York, NY 10003, USA 40 E 7th St, New York, NY 10003, USA |
| 7th Street | 838 Broadway, New York, NY 10003, USA |
| 838 Broadway | |
| 9th St. PATH Station Abu Dhabi House | 69 W 9th St, New York, NY 10011, USA |
| Affinia Manhattan Hotel | 19 Washington Square N, New York, NY 10011, USA |
| Alumni Hall | 371 7th Ave, New York, NY 10001 |
| Bobst Library Brittany Hall | 33 3rd Ave, New York, NY 10003, USA |
| Broome | 55 East 10th Street, New York, NY 10003, United States 400 Broome St, New York, NY 10013, USA |
| | 70 Washington Square South, New York, NY 10012, United States |
| Torch Club University Court University Hall Vanderbilt | |
| HKMC Kaufman Kevorkian Kimball Com Kimmel La Maison Lafayette Langone Medical Center Lipton Hall Mercer Res Wagner Grad Waverly Building | 29 Washington Pl, New York, NY 10003, USA 44 W 4th St, New York, NY 10012, USA 44 W 4th St, New York, NY 10012, USA 50 Washington Square S, New York, NY 10012, USA 246 Greene St, New York, NY 10003, USA 60 Washington Square S, New York, NY 10012, USA 16 Washington Mews, New York, NY 10003, USA 80 Lafayette St, New York, NY 10013, USA |
| Brown Bldg Cantor Film Carlyle Hall | |
| | 25 Union Square W, New York, NY 10003, USA |
| | 36 East 8th Street, New York, NY 10003, United States |
| Clark Residence Hall (Poly) | |
| Coles Center | |
| College of Dentistry | 55 Clark St, Brooklyn, NY 11201, USA |
| | 345 E 24th St, New York, NY 10010, USA |
| Computer Center Coral Tower | 181 Mercer Street, New York, NY 10012, United States |
| | 242 Greene St, New York, NY 10003, USA East 14th Street, New York, NY 10003, United States |
| D'Agostino East Bldg | 110 W 3rd St, New York, NY 10012, USA |
| Ehrenkranz | |
| Faculty of Arts | |
| | 13 Washington Square S, New York, NY 10012, USA |
| Weaver Weinstein | |
| West 4th St. MTA Station (W 3rd St. and 6th Ave) Wilf Hall | |
| | 136 W 3rd St, New York, NY 10012, USA |
| Furman Hall Gallatin Glucksman | 249 Sullivan St, New York, NY 10012, USA 715 Broadway, New York, NY 10003, USA 1 Washington Mews, New York, NY 10003, USA |
| Goddard Hall Gramercy Green Greenwich Hotel Health Services | 80 Washington Square E, New York, NY 10003, USA 310 3rd Ave, New York, NY 10010, USA 636 Greenwich St, New York, NY 10014, USA 726 Broadway, New York, NY 10003, USA |
| Founders Hall | 5 Washington Square S, New York, NY 10012, USA 120 E 12th St, New York, NY 10003, USA |
| | 239 Greene St, New York, NY 10003, USA |
| Pless Bldg Prince Juan Carlos Rubin Rufus School of Nursing Shimkin Silver Towers | 32 Washington Pl, New York, NY 10003, USA 7 Washington Pl, New York, NY 10003, USA Rubin Residence Hall, New York, NY 10003, USA |
| Public Safety Command Center | 53 Washington Square S, New York, NY 10012, USA |
| Meyer Hall Othmer Hall Palladium Pless Annex | 550 1st Ave, New York, NY 10016, USA 33 Washington Square W, New York, NY 10011, USA 240 Mercer St, New York, NY 10012, USA 4 Washington Pl, New York, NY 10003, USA 101 Johnson St, Brooklyn, NY 11201, USA 140 E 14th St, New York, NY 10003, USA 26 Washington Pl, New York, NY 10003, USA |
| | 25 Waverly Pl, New York, NY 10003, USA |
| | 433 1st Ave, New York, NY 10010, USA 50 W 4th St, New York, NY 10012, USA |
| | 100 Bleecker St, New York, NY 10013, USA |
| Steinhardt Stern | 35 W 4th St, New York, NY 10012, USA 40 W 4th St, New York, NY 10012, USA 238 1st Ave, New York, NY 10009, USA |
| Stuyvesant 14th St & 1st Ave Stuyvesant 14th St & Ave C | 690 E 14th St, New York, NY 10009, USA |
| Stuyvesant 20th St. & 1st Ave | |
| Stuyvesant 20th St. & Ave C | 339 1st Ave, New York, NY 10003, USA 610 E 20th St, New York, NY 10009, USA |
| Tikvah Tisch Hall | 40 W 4th St, New York, NY 10012, USA |
| Union Square MTA Station (Broadway and 14th St) | 22 Washington Square N, New York, NY 10011, USA |
| | 853 Broadway, New York, NY 10003, USA |
| | 18 Waverly Pl, New York, NY 10003, USA 334 E 25th St, New York, NY 10010, USA |
| | 120 E 14th St, New York, NY 10003, USA |
| | 40 Washington Square S, New York, NY 10012, USA 4 Washington Square N, New York, NY 10003, USA |
| | 24 Waverly Pl, New York, NY 10003, USA |
| | 251 Mercer St, New York, NY 10012, USA 5 University Pl, New York, NY 10003, USA |
| | 133 Macdougal St, New York, NY 10012, USA |
It doesn't seem to mistake lines for text. However, the layout is a bit off. It seems good for extracting text, but I got the impression that it's not suitable for converting extracted table data to CSV.
Conclusion
This time I tried Docling. From my impression so far, it seems to be good for extracting text information from PDFs, but it is not yet accurate enough to extract table data from images.
Also, I think it may be possible to adjust the accuracy by calling it from a command option or program. I would like to keep trying it.