I am succesfully processing my video into x264. I am happy with the output and file size I am generating, but the video seems to take as long as the video is to process.
For example, if the video is 10 seconds long, it will take 10 seconds to process the video, etc...
Even if I put -crf 50, it will take the same time. I find this odd behaviour.
Note: I am using FFmpeg with FFmpegKit (React Native): https://github.com/tanersener/ffmpeg-kit
I am using full-gpl in order to be able to encode to x264. I am not sure what is wrong here or if this is normal behaviour?
Here is my FFmpeg command I am executing:
`-y -i ${media.path} -c:v libx264 -preset veryfast -tune fastdecode -crf 20 -vf "crop=1350:1080, scale=960:780" -c:a copy -movflags faststart ${path}`
I appreciate all the help I can get here to speed this up / fix this issue.
Cheers!
I am trying to take the contents of a file that has a Hex number and convert that number to Binary and output to a file.
This is what I am trying but not getting the binary value:
xxd -r -p Hex.txt > Binary.txt
The contents of Hex.txt is: ff
I have also tried FF and 0xFF, but would like to just use ff since the device I am pulling the info from has it in that format.
Instead of 11111111 which it should be, I get a y with 2 dots above it.
If I change it to ee, I get an i with 2 dots. It seems to be reading it just fine but according to what I have read on the xxd -r -p command, it is not outputing it in the correct format.
The other ways I have found to convert Hex to Binary have either also not worked or is a pretty big Bash script that seems unnecessary to do what I thought would be a simple task.
This also gives me the y with 2 dots.
$ for i in $(cat Hex.txt) ; do printf "\x$i" ; done > Binary.txt
For some reason almost every solution I find gives me this format instead of a human readable Binary value with 1s and 0s.
Any help is appreciated. I am planning on using this in a script to pull the Relay values from Digital Loggers devices using curl and giving Home Assistant a readable file to record the Relay State. Digital Loggers curl cmd gives the state of all 8 relays at once using Hex instead of being able to pull the status of a specific relay.
If "file.txt" contains:
fe
0a
and you run this:
perl -ane 'printf("%08b\n",hex($_))' file.txt
You'll get this:
11111110
00001010
If you use it a lot, you might want to make a bash function of it in your login profile along these lines - being extremely respectful of spaces and semi-colons that might look unnecessary:
bin(){ perl -ane 'printf("%08b\n",hex($_))' $1 ; }
Then you'll be able to do:
bin file.txt
If you dislike Perl for some reason, you can achieve something similar without it as follows:
tr '[:lower:]' '[:upper:]' < file.txt |
while read h ; do
echo "obase=2; ibase=16; $h" | bc
done
I'm working on a project in which I need to extract a TIFF per page from multi-page PDFs. The PDFs contain images only and there is one image per page (I believe they were made on some kind of photocopier/scanner, but haven't confirmed this). The TIFFs are then used to create several other derivative versions of the document so the higher the resolution the better.
I've found two recipes, both with helpful aspects, but neither is ideal. Hoping someone can help me tune one of them, or offer a third option.
Recipe 1, pdfimages and ImageMagick:
First do:
$ pdfimages $MY_PDF.pdf foo"
Which results in several .pbm files (named foo-000.pbm, foo-001.pbm), etc.
Then for each *.pbm do:
$ convert $each -resize 3200x3200\> -quality 100 $new_name.tif
Pro: The resultant TIFFs are a healthy 3300+ pixels on the long dimension, (-resize just serves to normalize everything)
Con: The orientation of the pages is lost, and they come out rotated different directions (they follow logical patterns, so probably they are the orientation in which they were fed to the scanner??).
Recipe 2 Imagemagick solo:
convert +adjoin $MY_PDF.pdf pages.tif
This gives me a TIFF per page (pages-0.tif, pages-1.tif, etc.).
Pro: Orientation stays!
Con: The long dimension of the resultant file is < 800 px, which is too small to be useful, and it looks as though there is some compression applied.
How can I ditch the scaling of the image stream in the PDF, but retain the orientation? Is there some more magick in ImageMagick that I'm missing? Something else entirely?
Sorry for the noise on this old topic, but google took me here as one of the top results and it might take others, so I thought I'd post the solution for the TO's question that I found here: http://robfelty.com/2008/03/11/convert-pdf-to-png-with-imagemagick
In Short: You have to tell ImageMagick at which density it should scan the PDF.
so convert -density 600x600 foo.pdf foo.png will tell ImageMagick to treat the PDF as if it had a 600dpi resolution and thus output much larger PNGs. In my case, the resulting foo.png was sized 5000x6600px. You can optionally add -resize 3000x3000 or whatever size you require and it will be scaled down.
Note that as long as you only have vector images or text in your PDF-files, density might be set as high as needed. If the PDF contains rasterized images, it won't look good if you set it higher than those images' dpi, surprise! :)
Chris
I wanted to share my solution...it may not work for everyone, but since nothing else has come around maybe it will help someone else. I wound up going with the first option in my question, which was to use pdfimages to get large images that were rotated every which way. I then found a way to use OCR and word counts to guess at the orientation, which got me from (estimated) 25% rotated accurately to above 90%.
The flow is as follows:
Use pdfimages (apt-get install poppler-utils) to get a set of pbm
files (not shown below).
For each file:
Make four versions, rotated 0, 90, 180, and 270 degrees (I refer to them as "north", "east", "south", and "west" in my code).
OCR each. The two with the lowest word count are likely the right-side up and upside down versions. This was over 99% accurate in my set of images processed to date.
From the two with the lowest word count, run the OCR output through a spell check. The file with the least spelling errors (i.e. most recognizable words) is likely to be correct. For my set this was about 93% (up from 25%) accurate based on a sample of 500.
YMMV. My files are bitonal and highly textual. The source images are an average of 3300 px on the long side. I can't speak to greyscale or color, or files with a lot of images. Most of my source PDFs are bad scans of old photocopies, so the accuracy might be even better with cleaner files. Using -despeckle during the rotation made no difference and slowed things down considerably (~5×). I chose ocrad for speed and not accuracy since I only need rough numbers and am throwing away the OCR. Re: performance, my nothing-special Linux desktop machine can run the whole script over about 2-3 files/per second.
Here's the implementation in a simple bash script:
#!/bin/bash
# Rotates a pbm file in place.
# Pass a .pbm as the only arg.
file=$1
TMP="/tmp/rotation-calc"
mkdir $TMP
# Dependencies:
# convert: apt-get install imagemagick
# ocrad: sudo apt-get install ocrad
ASPELL="/usr/bin/aspell"
AWK="/usr/bin/awk"
BASENAME="/usr/bin/basename"
CONVERT="/usr/bin/convert"
DIRNAME="/usr/bin/dirname"
HEAD="/usr/bin/head"
OCRAD="/usr/bin/ocrad"
SORT="/usr/bin/sort"
WC="/usr/bin/wc"
# Make copies in all four orientations (the src file is north; copy it to make
# things less confusing)
file_name=$(basename $file)
north_file="$TMP/$file_name-north"
east_file="$TMP/$file_name-east"
south_file="$TMP/$file_name-south"
west_file="$TMP/$file_name-west"
cp $file $north_file
$CONVERT -rotate 90 $file $east_file
$CONVERT -rotate 180 $file $south_file
$CONVERT -rotate 270 $file $west_file
# OCR each (just append ".txt" to the path/name of the image)
north_text="$north_file.txt"
east_text="$east_file.txt"
south_text="$south_file.txt"
west_text="$west_file.txt"
$OCRAD -f -F utf8 $north_file -o $north_text
$OCRAD -f -F utf8 $east_file -o $east_text
$OCRAD -f -F utf8 $south_file -o $south_text
$OCRAD -f -F utf8 $west_file -o $west_text
# Get the word count for each txt file (least 'words' == least whitespace junk
# resulting from vertical lines of text that should be horizontal.)
wc_table="$TMP/wc_table"
echo "$($WC -w $north_text) $north_file" > $wc_table
echo "$($WC -w $east_text) $east_file" >> $wc_table
echo "$($WC -w $south_text) $south_file" >> $wc_table
echo "$($WC -w $west_text) $west_file" >> $wc_table
# Take the bottom two; these are likely right side up and upside down, but
# generally too close to call beyond that.
bottom_two_wc_table="$TMP/bottom_two_wc_table"
$SORT -n $wc_table | $HEAD -2 > $bottom_two_wc_table
# Spellcheck. The lowest number of misspelled words is most likely the
# correct orientation.
misspelled_words_table="$TMP/misspelled_words_table"
while read record; do
txt=$(echo $record | $AWK '{ print $2 }')
misspelled_word_count=$(cat $txt | $ASPELL -l en list | wc -w)
echo "$misspelled_word_count $record" >> $misspelled_words_table
done < $bottom_two_wc_table
# Do the sort, overwrite the input file, save out the text
winner=$($SORT -n $misspelled_words_table | $HEAD -1)
rotated_file=$(echo $winner | $AWK '{ print $4 }')
mv $rotated_file $file
# Clean up.
if [ -d $TMP ]; then
rm -r $TMP
fi
Hallo.
I have a big video file. ffmpeg, tcprobe and other tool say, it is an h264-stream in an AVI-container.
Now i'd like to cut out small chunks form the video.
Problem: The index of the video seam corrupted/destroyed. I kind of fixed this via mplayer -forceidx -saveidx <IndexFile> <BigVideoFile>. The Problem here is, that I'm now stuck with mplayer/mencoder which can use this index file via -loadidx <IndexFile>. I have tried correcting the index like described in man aviindex (mplayer -frames 0 -saveidx mpidx broken.avi ; aviindex -i mpidx -o tcindex ; avimerge -x tcindex -i broken.avi -o fixed.avi), but this didn't fix my video - meaning that most tools i've tested couldn't search in the video file.
Problem: I cut out parts of the video via following command: mencoder -loadidx in.idx -ss 8578 -endpos 20 -oac faac -ovc x264 -sws 9 -lavfopts format=mp4 -x264encopts <LotsOfOpts> -of lavf -vf scale=800:-10,harddup in.avi -o out.mp4. Now here the problem is, that some videos are corrupted at the beginning. I think this is because the fact, that i do not necessarily cut at keyframe.
Questions:
What is the best way to fix the index of an avi "inline" so that every tool can again work as expected with it?
How can i split at the keyframes? Is there an mencoder-option for this?
Are Keyframes coming in a frequency? How to find out this frequency? (So with a bit of math it should be possible to calculate the next keyframe and cut there)
Is ther perhaps some completely other way to split this movie? Doing it by hand is no option, i've to cut out 1000+ chunks ...
Thanks a lot!
https://spreadsheets.google.com/ccc?key=0AjWmZ0umsuZHdHNzZVhuMTkxTHdYbUdCQzF3cE51Snc&hl=en lists the various options available for splitting accurately.
I would attempt to use avidemux to repair the file before doing anything. You may also have better results using an MP4 Based Container than AVI.
As for ensuring your specified intervals are right at the keyframe, I would suggest encoding with FFMPEG with the: -g 1 option before using the split below to ensure every frame is in fact a keyframe. FFMPEG refers to keyframs as GOP or Groups of Pictures instead.
ffmpeg -i input.avi -g 1 -vcodec copy -acodec copy out.avi
Then multiple splits (with FFMPEG) :
ffmpeg -i input.avi -ss 00:00:10 -t 00:00:30 out1.avi -ss 00:00:35 -t 00:00:30 out2.avi
Some more options to try:
x264 Mapping FFMPEG encoding in linux
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 9 years ago.
Improve this question
I've just introduced a friend to GNU Screen and they're having a hard time getting used to it. That makes me think about the essential things he needs to know about the excellent Screen utility, the same things that you'd think worthwhile to teach someone, a beginner, from the ground up. What are some analogies and handy tips for remembering binds, etc.?
It would be awesome.
I've been using Screen for over 10 years and probably use less than half the features. So it's definitely not necessary to learn all its features right away (and I wouldn't recommend trying). My day-to-day commands are:
^A ^W - window list, where am I
^A ^C - create new window
^A space - next window
^A p - previous window
^A ^A - switch to previous screen (toggle)
^A [0-9] - go to window [0-9]
^A esc - copy mode, which I use for scrollback
I think that's it. I sometimes use the split screen features, but certainly not daily. The other tip is if screen seems to have locked up because you hit some random key combination by accident, do both ^Q and ^A ^Q to try to unlock it.
I couldn't get used to screen until I found a way to set a 'status bar' at the bottom of the screen that shows what 'tab' or 'virtual screen' you're on and which other ones there are. Here is my setup:
[roel#roel ~]$ cat .screenrc
# Here comes the pain...
caption always "%{=b dw}:%{-b dw}:%{=b dk}[ %{-b dw}%{-b dg}$USER%{-b dw}#%{-b dg}%H %{=b dk}] [ %= %?%{-b dg}%-Lw%?%{+b dk}(%{+b dw}%n:%t%{+b dk})%?(%u)%?%{-b dw}%?%{-b dg}%+Lw%? %{=b dk}]%{-b dw}:%{+b dw}:"
backtick 2 5 5 $HOME/scripts/meminfo
hardstatus alwayslastline "%{+b dw}:%{-b dw}:%{+b dk}[%{-b dg} %0C:%s%a %{=b dk}]-[ %{-b dw}Load%{+b dk}:%{-b dg}%l %{+b dk}] [%{-b dg}%2`%{+b dk}] %=[ %{-b dg}%1`%{=b dk} ]%{-b dw}:%{+b dw}:%<"
sorendition "-b dw"
[roel#roel ~]$ cat ~/scripts/meminfo
#!/bin/sh
RAM=`cat /proc/meminfo | grep "MemFree" | awk -F" " '{print $2}'`
SWAP=`cat /proc/meminfo | grep "SwapFree" | awk -F" " '{print $2}'`
echo -n "${RAM}kb/ram ${SWAP}kb/swap"
[roel#roel ~]$
Ctrl+A ? - show the help screen!
If your friend is in the habit of pressing ^A to get to the beginning of the line in Bash, he/she is in for some surprises, since ^A is the screen command key binding. Usually I end up with a frozen screen, possibly because of some random key I pressed after ^A :-)
In those cases I try
^A s and ^A q block/unblock terminal scrolling
to fix that. To go to the beginning of a line inside screen, the key sequence is ^A a.
You can remap the escape key from Ctrl + A to be another key of your choice, so if you do use it for something else, e.g. to go to the beginning of the line in bash, you just need to add a line to your ~/.screenrc file. To make it ^b or ^B, use:
escape ^bB
From the command line, use names sessions to keep multiple sessions under control. I use one session per task, each with multiple tabs:
screen -ls # Lists your current screen sessions
screen -S <name> # Creates a new screen session called name
screen -r <name> # Connects to the named screen sessions
When using screen you only need a few commands:
^A c Create a new shell
^A [0-9] Switch shell
^A k Kill the current shell
^A d Disconnect from screen
^A ? Show the help
An excellent quick reference can be found here. It is worth bookmarking.
Some tips for those sorta familiar with screen, but who tend to not remember things they read in the man page:
To change the name of a screen window is very easy: ctrl+A shift+A.
Did you miss the last message from screen? ctrl+a ctrl+m will show it again for you.
If you want to run something (like tailing a file) and have screen tell you when there's a change, use ctrl+A shift+m on the target window. Warning: it will let you know if anything changes.
Want to select window 15 directly? Try these in your .screenrc file:
bind ! select 11
bind # select 12
bind \# select 13
bind $ select 14
bind % select 15
bind \^ select 16
bind & select 17
bind * select 18
bind ( select 19
bind ) select 10
That assigns ctrl+a shift+0 through 9 for windows 10 through 19.
Ctrl+A is the base command
Ctrl+A N = go to the ***N***ext screen
Ctrl+A P = go to the ***P***revious screen
Ctrl+A C = ***C***reate new screen
Ctrl+A D = ***D***etach your screen
http://www.debian-administration.org/articles/34
I wrote that a couple of years ago, but it is still a good introduction that gets a lot of positive feedback.
I "must" add this: add
bind s
to your .screenrc, if You - like me - used to use split windows, as C-a S splits the actual window, but C-a s freezes it. So I just disabled the freeze shortcut.
Ctrl+a is a special key.
Ctrl+a d - [d]etach, leave programs (irssi?) in background, go home.
Ctrl+a c [c]reate a new window
Ctrl+a 0-9 switch between windows by number
screen -r - get back to detached session
That covers 90% of use cases. Do not try to show all the functionality at the single time.
Not really essential not solely related to screen, but enabling 256 colors in my terminal, GNU Screen and Vim improved my screen experience big time (especially since I code in Vim about 8h a day - there are some great eye-friendly colorschemes).
The first modification I make to .screenrc is to change the escape command. Not unlike many of you, I do not like the default Ctrl-A sequence because of its interference with that fundamental functionality in almost every other context. In my .screenrc file, I add:
escape `e
That's backtick-e.
This enables me to use the backtick as the escape key (e.g. to create a new screen, I press backtick-c, detach is backtick-d, backtick-? is help, backtick-backtick is previous screen, etc.). The only way it interferes (and I had to break myself of the habit) is using backtick on the command line to capture execution output, or pasting anything that contains a backtick. For the former, I've modified my habit by using the BASH $(command) convention. For the latter, I usually just pop open another xterm or detach from screen then paste the content containing the backtick. Finally, if I wish to insert a literal backtick, I simply press backtick-e.
There is some interesting work being done on getting a good GNU screen setup happening by default in the next version of Ubuntu Server, which includes using the bottom of the screen to show all the windows as well as other useful machine details (like number of updates available and whether the machine needs a reboot). You can probably grab their .screenrc and customise it to your needs.
The most useful commands I have in my .screenrc are the following:
shelltitle "$ |bash" # Make screen assign window titles automatically
hardstatus alwayslastline "%w" # Show all window titles at bottom line of term
This way I always know what windows are open, and what is running in them at the moment, too.
I use the following for ssh:
#!/bin/sh
# scr - Runs a command in a fresh screen
#
# Get the current directory and the name of command
wd=`pwd`
cmd=$1
shift
# We can tell if we are running inside screen by looking
# for the STY environment variable. If it is not set we
# only need to run the command, but if it is set then
# we need to use screen.
if [ -z "$STY" ]; then
$cmd $*
else
# Screen needs to change directory so that
# relative file names are resolved correctly.
screen -X chdir $wd
# Ask screen to run the command
if [ $cmd == "ssh" ]; then
screen -X screen -t ""${1##*#}"" $cmd $*
else
screen -X screen -t "$cmd $*" $cmd $*
fi
fi
Then I set the following bash aliases:
vim() {
scr vim $*
}
man() {
scr man $*
}
info() {
scr info $*
}
watch() {
scr watch $*
}
ssh() {
scr ssh $*
}
It opens a new screen for the above aliases and iff using ssh, it renames the screen title with the ssh hostname.
I like to set up a screen session with descriptive names for the windows. ^a A will let you give a name to the current window and ^a " will give you a list of your windows.
When done, detach the screen with ^a d and re-attach with screen -R
I like to use screen -d -RR to automatically create/attach to a given screen. I created bash functions to make it easier...
function mkscreen
{
local add=n
if [ "$1" == '-a' ]; then
add=y
shift;
fi
local name=$1;
shift;
local command="$*";
if [ -z "$name" -o -z "$command" ]; then
echo 'Usage: mkscreen [ -a ] name command
-a Add to .bashrc.' 1>&2;
return 1;
fi
if [ $add == y ]; then
echo "mkscreen $name $command" >> $HOME/.bashrc;
fi
alias $name="/usr/bin/screen -d -RR -S $name $command";
return 0;
}
function rmscreen
{
local delete=n
if [ "$1" == '-d' ]; then
delete=y
shift;
fi
local name=$1;
if [ -z "$name" ]; then
echo 'Usage: rmscreen [ -d ] name
-d Delete from .bashrc.' 1>&2;
return 1;
fi
if [ $delete == y ]; then
sed -i -r "/^mkscreen $name .*/d" $HOME/.bashrc;
fi
unalias $name;
return 0;
}
They create an alias to /usr/bin/screen -d -RR -S $name $command. For example, I like to use irssi in a screen session, so in my .bashrc (beneath those functions), I have:
mkscreen irc /usr/bin/irssi
Then I can just type irc in a terminal to get into irssi. If the screen 'irc' doesn't exist yet then it is created and /usr/bin/irssi is run from it (which connects automatically, of course). If it's already running then I just reattach to it, forcibly detaching any other instance that is already attached to it. It's quite nice.
Another example is creating temporary screen aliases for perldocs as I come across them:
mkscreen perlipc perldoc perlipc
perlipc # Start reading the perldoc, ^A d to detach.
...
# Later, when I'm done reading it, or at least finished
# with the alias, I remove it.
rmscreen perlipc
The -a option (must be first argument) appends the screen alias to .bashrc (so it's persistent) and -d removes it (these can potentially be destructive, so use at own risk). xD
Append:
Another bash-ism that I find convenient when working a lot with screen:
alias sls='/usr/bin/screen -ls'
That way you can list your screens with a lot fewer keystrokes. I don't know if sls collides with any existing utilities, but it didn't at the time on my system so I went for it.
^A A switches back to the screen you just came from.
Ctrl + A is a great special character for Unix people, but if you're using screen to talk to OpenVMS, then not being able to ^A is going to make you bald prematurely.
In VMS, if you're editing a DCL command prior to execution from the history buffer, Insert mode is off (it has to be for a few reasons I won't get into here) ... to turn it on so you don't over-type your command rather than space things out, you have to hit `^A.