Exploring Web Scraping with Microsoft's Markitdown Tool

This should be fun to play with for anyone messing with web scraping or rag https://github.com/microsoft/markitdown

· Sorted by Oldest

This should be fun to play with for anyone messing with web scraping or rag https://github.com/microsoft/markitdown

· Sorted by Oldest

Anthony P.
·
PDF (.pdf)
PowerPoint (.pptx)
Word (.docx)
Excel (.xlsx)
Images (EXIF metadata, and OCR)
Audio (EXIF metadata, and speech transcription)
HTML (special handling of Wikipedia, etc.)
Various other text-based formats (csv, json, xml, etc.)
ZIP (Iterates over contents and converts each file)
Anthony P.
·
The current list of supported document types is nice
Xander S.
·
I have marked it down
Xander S.
·
✍️
Mikyo
·
Weirdly I've never questioned why LLMs prefer markdown / XML. I'm guessing it's based on pre-training but not sure I actually know why.

Anthony P.
·
PDF (.pdf)
PowerPoint (.pptx)
Word (.docx)
Excel (.xlsx)
Images (EXIF metadata, and OCR)
Audio (EXIF metadata, and speech transcription)
HTML (special handling of Wikipedia, etc.)
Various other text-based formats (csv, json, xml, etc.)
ZIP (Iterates over contents and converts each file)
Anthony P.
·
The current list of supported document types is nice
Xander S.
·
I have marked it down
Xander S.
·
✍️
Mikyo
·
Weirdly I've never questioned why LLMs prefer markdown / XML. I'm guessing it's based on pre-training but not sure I actually know why.