export entire pelican site as pdf? - pdf

Is there any easy way to export an entire Pelican site as pdf, sorted by category?
I tried dragging a sitemap html into calibre but this sucks in the navigation too, which I don't want.
It would be easy to extract only the content from the html pages in the output folder with say python but then the pages are not sorted by category.
I could convert the .md files to pdf with pandoc but I have planturl images created from the markdown by a plugin so this wouldn't work either because the pdf wouldn't have the graphics.
Any ideas?

Hacky homerolled solution: - I wrote extractDivs.py https://gist.github.com/stevepowell99/335ef06fe2b35b9317ed to go through the output folder and just extract the article div from each post, then put this in my makefile, after pelican runs:
python extractDivs.py
pandoc -s -S output/all.html -o output/book.html -c ../pelican-themes/monospace/static/css/main.css
wkhtmltopdf --footer-right [page] --footer-left [section] -R 30 -L 30 -T 30 -B 30 --outline toc ./output/book.html ./output/book.pdf
so pandoc takes the extracted divs and puts them into a single file with the same css as my cite, and then wkhtmltopdf converts to pdf.
Yuk.

The best place to start is probably the PDF plugin. The README only mentions reST, so I'm not sure if Markdown is supported, but if not it could probably be extended to do so.

Related

How can I convert specific web page to markdown or asciidoc with pandoc?

I want to convert java specification documentation to easily editable formats(markdown or asciidoc) and upload GitHub Gist and customize(adding my code experiences and notes.)
I want to convert to something like this
I use a tool called pandoc that allows us to convert from HTML to markdown.
I tried followings:
Technique 1
I tried to convert all table of components of java specification on index.html
pandoc -f html -t markdown -o test2.md
https://docs.orac le.com/javase/specs/jls/se10/html/index.html`
I got this:tes2.md
(I did not upload here because the file of contents is too long)
Problem 1:
This markdown file does not have contents of java specification documentation. I expected that I got markdown toc(table of components) and java specification documentation contents in markdown file like this`
Problem 2:
When click the links on this markdown file then I get 404 error page.
Technique 2(Better than technique 1)
I downloaded all HTML files of TOC with HTTrack and try to convert all files separately.
pandoc -f html-native_divs-native_spans -i jls-1.html -t markdown -o test2.md
Problem 1:
I got following markdown file which have the table of components links that cannot redirect to another section of the same document. When I click on this links, they return external GitHub page like that:https://gist.github.com/lostdinar2/jls-1.html#jls-1.1
which is not available.
test3.md
A demonstration of problem 1:
1)I want to convert this HTML internal id link(#) to the markdown internal link that redirects to another section of the same document
<dt><span class="section">2.2. The Lexical Grammar</span></dt>
[link text](#abcd)
2)But pandoc cannot convert this links to the markdown internal link.Pandoc create an external link like this:https://gist.github.com/lostdinar2/jls-1.html#jls-1.1
Is there a pandoc parameter to fix this? I make a search on pandoc documentation but I cannot do this feature.

How to avoid automatic appended file extensions to directory links when converting to pdf using pandoc?

I'm writing company internal documentation in R markdown and compiling using knitr in Rstudio. I'm trying to add a link pointing to a directory as follows:
[testdir](file:////c:/test/)
(this is following the convention described in here)
When I compile it to html, I get the following link.
testdir
and it works as expected in Internet explorer. However, when I try to convert to pdf straight from RStudio, an unwanted pdf extension is appended to the link. I tried dissecting the problem and it seems this change is happening within pandoc. Here are the details.
When I convert it to latex using pandoc,
>pandoc -f markdown -t latex testing.md -o test.tex
the link in the latex output file looks as follows:
\href{file:///c:/test/}{testdir}
Everything good so far. However, when I convert the latex output to pdf with pandoc,
>pandoc -f latex -t latex -o test.pdf test.tex
a .pdf extension is appended to the link. Here is a copy/paste of the pdf link output:
/c:/test/.pdf
is there a way to avoid this unwanted appended extension?
Perhaps I'm asking too much of pandoc, but I thought it might be worth asking since RStudio is becoming such a useful IDE to write my dynamic documents.
As you said, the .tex file pandoc generates is fine. So the problem is actually with LaTeX, specifically with the hyperref package which is used in pandoc's LaTeX template.
The problem with two possible solutions was described here. To prevent hyperref from being smart and adding a file extensions, try:
[testdir](file:///c:/test/.)
Or use ConTeXt instead of LaTeX:
$ pandoc -t context -s testing.md -o test.tex && context test.tex

Pandoc markdown to PDF

I'm looking to convert my Markdown book to a PDF.
I've done a lot of research and it seems that Pandoc is the best choice for this. It seems pandoc converts the markdown to latex and then to a PDF.
The problem I am running into is including external images. Ideally I would have the process grab the remote images off the net and put them into the pdf.
I'm hitting this error:
pandoc: Error producing PDF from TeX source.
! Package pdftex.def Error: File `http://wes.io/QYGG/content.png' not found.
See the pdftex.def package documentation for explanation.
Type H <return> for immediate help.
...
l.84 ...degraphics{http://wes.io/QYGG/content.png}
I have MacTex installed and the command I'm running is pandoc test.md -o test.pdf
I've never used latex before, so I'm a bit at a loss of how to fix this.
According to this LaTex question, you can't directly reference URL images from within LaTeX, though they have a potential LaTeX-hacking option available at the link.
Local image files are probably your best bet.
Newer versions of pandoc fetch external images before passing them on to LaTeX.
Alternatively, you could use ConTeXt instead of LaTeX which natively supports fetching images from URLs:
$ pandoc -t context test.md -o test.pdf

Effortless export from github wiki (?)

I am collecting quite a lot of material in a GitHub wiki. I really like to use the wiki to cooperate with other people and IMHO the platform is really nice, I like it!
So, I would like to keep using the GH wiki to collect stuff, edit, save,etc but I also would like to export the content in order to create a pdf file that we can call "a manual".
I would like to generate an updated version of the manual automatically everytime I want just running a couple of scripts, I can not put too much effort on this.
I guess it is possible to export the content somehow and the use pandoc (http://johnmacfarlane.net/pandoc/) to create the pdf maybe adding an index and a style file.
Another interesting idea could be publish a website once a month dumping content directly from the wiki.
I guess other people already did something like this but I did not find anynthing.
Any idea?
But... the Github wiki of a GitHub repo is a git repo in itself (introduced in August 2010).
You can clone it, push to it or pull from it.
Each wiki is a Git repository, so you're able to push and pull them like anything else.
Each wiki respects the same permissions as the source repository.
Just add ".wiki" to any repository name in the URL, and you're ready to go.
Or, as noted by htafoya in the comments, replace the .git part of the URL (if present) by .wiki.
That makes the "export" part of your question really trivial.
From there, you will find tons of script for converting markdown pages into pdf:
a graddle task
a makefile
a python script
...
I'm adding to this answer, in case it helps any new readers :) here's what I did:
I installed GitHub Desktop: https://desktop.github.com/
Then, on the wiki page in my repository, I clicked "Clone in Desktop"
This saved the wiki locally as a .md file (after following the steps on screen)
I then used http://www.markdowntopdf.com/ to convert it to pdf
(Note: I renamed the files to remove characters that wouldn't work in a pdf file name before uploading to the website)
The end result was really nice.
I found many of the solutions difficult to reproduce/get the right version/understand/fix/etc... So instead, I'll present a patchwork docker solution to effortlessly convert on Windows(using git bash)/MacOS/Linux in 5 "easy" commands
git clone {project_url}.wiki .
# Convert *.md to *.md.html using the actual github pipeline
docker run --rm -e DOCKER_USER_ID=`id -u` -e DOCKER_GROUP_ID=`id -u` \
v "`pwd`:/src" -v "`pwd`:/out" andyneff/github-markdown-preview
# Fix hyperlinks, since wkhtmltopdf is stricter than github servers
docker run --rm -v `pwd`:/src -w /src perl \
perl -p -i -e 's|(.*?)|\1\L\2\E.md.html\L\3\E\4|g'\
*.html
# Lowercase all filename so that hyperlink match
docker run --rm -v `pwd`:/src -w /src python \
python -c 'import sys;import os; [os.rename(f, f.lower()) for f in sys.argv[1:]]' \
*.md.html
#Convert html to pdf using QT webkit
docker run -it --rm -e DOCKER_USER_ID=`id -u` -e DOCKER_GROUP_ID=`id -u`\
-v `pwd`:/work -w /work andyneff/wkhtmltopdf \
wkhtmltopdf --encoding utf-8 --minimum-font-size 14 \
--footer-left "[date]" --footer-right "[page] / [topage]" \
--footer-font-size 10 \
toc \
*.html document.pdf
The perl is the main part that may fail without a better solution. Pandoc has a really good filter solution, but isn't using the github pipeline.
Bugs:
Extra wide code blocks will be rendered with a scroll bar, and essentially cut off in the pdf. It would be best to make the code block not overflow, but you can add --user-style-sheet user.css to the wkhtmltopdf command (before toc/cover), and add to your user.css
.markdown-body .highlight pre,
.markdown-body pre{
overflow:visible !important;
}
Some link in the final pdf are off by +1 page, some are not. Not sure what the pattern is. But anchors with ids (#) do not appear to have this problem
Another option once you clone the wiki, especially if you are already using Atom is to use this Markdown to PDF package.
Worked great for me.
I found really annoying having to convert each markdown document separately (links between markdown documents are lost), so I ended up writting a simple C# program for my own use that does this in a single step: a) Download the last version of the wiki from Github, b) Convert it all the markdown documents merged as one pdf
You can download the binaries (Windows or any platform supporting Mono) from:
https://github.com/borjafdezgauna/CoderDocTools/releases/latest
If, for example, you want to convert to PDF the SimionZoo repository by user simionsoft, you can:
MarkdownToPDF.exe user=simionsoft project=SimionZoo output-file=SimionZoo.pdf
I've accomplished precisely this when creating the portable documentation for Barcode Writer in Pure PostScript:
GitHub Wiki + Makefile + pandoc → PDF
The process is described in this blog post.
This question has already been answered but wanted to add my quick experience here.
I didn't find it necessary to install the Desktop version of Github. You can clone by simply running the following from your commandline:
git clone git#github.com:<username>/<repository>.wiki.git
(Of course, replace username and repository as needed).
The cloned wiki outputted 72 markdown files. As has been previously said, there are numerous ways of converting these files do PDF, you can pick your own tool. However I will say that the easiest solution I encountered was to install Pandoc. I have macOS + homebrew, so a quick brew install pandoc was all I needed.
Some info on using pandoc here: https://stackoverflow.com/a/14908316/3638172
You can also try html_links_to_pdf!
It's a Python 3 script made just to convert a GitHub Wiki to pdf form, using the same styling that GitHub uses, but slightly cleaner.

How to convert a Markdown file to PDF

I have a Markdown file that I wish to convert to PDF so that I can upload it on Speakerdeck. I am using Pandoc to convert from markdown to PDF.
My problem is I can't specify what content should go on what page of the PDF, because Markdown doesn't provide any feature like that.
E.g., Markdown:
###Hello
* abc
* def
###Bye
* ghi
* jkl
Now I want Hello to be one slide and Bye to be on another slide on Speakerdeck. So, I will need them to be on different pages in the PDF that I generate using Pandoc.
But both Hello and Bye gets on the same page in the PDF.
How can I accomplish this?
Via the terminal (tested in 2020)
Download dependencies
sudo apt-get install pandoc texlive-latex-base texlive-fonts-recommended texlive-extra-utils texlive-latex-extra
Try to use
pandoc MANUAL.txt -o example13.pdf
pandoc MANUAL.md -o example13.pdf
Via a Visual Studio Code extension (tested in 2020)
Download the Yzane Markdown PDF extension
Right click inside a Markdown file (md)
The content below will appear
Select the Markdown PDF: Export (pdf) option
Note: Emojis are better in Windows than Linux (I don't know why)
2016 update:
NPM module: https://github.com/alanshaw/markdown-pdf
Has a command line interface: https://github.com/alanshaw/markdown-pdf#usage
npm install -g markdown-pdf
markdown-pdf <markdown-file-path>
Or, an online service: http://markdown2pdf.com
As SpeakerDeck only accepts PDF files, the easiest option is to use the Latex Beamer backend for pandoc:
pandoc -t beamer -o output.pdf yourInput.mkd
Note that you should have LaTeX Beamer installed for that.
In Ubuntu, you can do sudo apt-get install texlive-latex-recommended to install it. If you use Windows, you may try this answer.
You may also want to try the HTML/CSS output from Slidy:
pandoc --self-contained -t slidy -o output-slidy.html yourInput.mkd
It has a decent printing output, as you can check out trying to print the original.
Read more about slideshows with pandoc here.
Easy online solution: dillinger.io.
Just paste your Markdown content into the editor on the left and see the (html) preview on the right. Then click Export as on the top and chose pdf.
It's based on the open source dillinger editor.
Adding to elias' answer, if you want to separate text in slides, just put *** between the text you want to separate. For your example to be in several pages, write it like this:
### Hello
- abc
- def
***
### Bye
- ghi
- jkl
And then use elias' answer, pandoc -t beamer -o output.pdf yourInput.md.
I have Ubuntu 18.10 (Cosmic Cuttlefish) and installed the full package from texlive. It works for me.
Previously I had used the npm markdown-pdf answer. However, on a fresh install of Ubuntu 19.04 (Disco Dingo) I had issues getting it to install correctly.
Instead I started using the Visual Studio Code package: "Markdown PDF"
Details:
Name: Markdown PDF
Id: yzane.markdown-pdf
Description: Convert Markdown to PDF
Version: 1.2.0
Publisher: yzane
Visual Studio Marketplace link: https://marketplace.visualstudio.com/items?itemName=yzane.markdown-pdf
It has worked consistently well. If you've had issues getting other answers to work, I would recommend trying this.
I've managed to get a stable Markdown -> HTML > PDF pipeline working with the MarkReport project.
It is a bit more than what Pandoc will do though, since it is based on WeasyPrint and is therefore aimed for clean report publishing, with cover, headers, sections, ...
It also enriches the HTML with syntax highlighting and LaTeX equations.
Simple way with iOS:
Use Shortcuts app (by Apple)
Make Rich Text From Markdown: Clipboard
^
Make PDF from Rich Text From Markdown
^
Show [PDF] in Quick Look
Just copy text and run the shortcut. Press share in Quick Look (bottom left) to store or send it. I use this to quickly convert Joplin notes to pdf.
I found that many markdown-to-pdf converters produce files that I don't find exactly neat-looking. However there is a solution to this.
If you're using IntelliJ, you can use a plugin called "Markdown". The export function uses pandoc as an engine so you probably will need to install that along with pdf-latex. https://pandoc.org/installing.html
In IntelliJ, under Tools > Markdown Converter > Export Markdown File To...
And there you go, a clean looking document. Additional styling can be added via a .css stylesheet.