I currently am trying to migrate the following command from ffprobe to ffmpeg-python
ffprobe -v error -select_streams v:0 -count_packets -show_entries stream=nb_read_packets -of csv=p=0 input.mp4
I tried the following and it did not work
ffmpeg.probe(file_path, v='error', select_streams='v:0', count_packets=True, show_entries=True, stream='nb_read_packets', of=True, csv='p=0')
I have tried this inputting it as a dictionary as well
ffmpeg.probe(file_path, **{'v':'error', 'select_streams':'v:0', 'count_packets':1, 'show_entries':1, 'stream':'nb_read_packets', 'of':'csv=p=0'})
But none of these seem to work, any help would be appreciated
TL;DR Answer
With the way the parameters for the python bindings for ffmpeg-python work, the format of the command you are inputting is incorrect. Instead the following should work for you
import ffmpeg, json
out = ffmpeg.probe("input.mp4", cmd='ffprobe', v='error', select_streams='v:0', count_packets=None, show_entries='stream=nb_read_packets')
out["streams"][0]["nb_read_packets"]
Complete Answer
Digging through the API reference for the ffmpeg-python module, the python binding ffmpeg.probe, states that it will
Run ffprobe on the specified file and return a JSON representation of
the output.
Debugging through the module/source code, this python binding for ffprobe will always add the command line options -show_format, -show_streams, -of json to your command that you pass in as defined in the
_probe.py file.
So for the ffprobe command you provided:
ffprobe -v error -select_streams v:0 -count_packets -show_entries stream=nb_read_packets -of csv=p=0 input.mp4
will essentially be this command line options after passing through ffmpeg.fprobe:
ffprobe -show_format -show_streams -v error -select_streams v:0 -count_packets -show_entries stream=nb_read_packets -of csv=p=0 -of json input.mp4
I downloaded an mp4 video that was probably a chromecast commercial based on the description. Here's the gist link that I found it from, and look for the title "For Bigger Escape". I downloaded the file and renamed it input.mp4 to match the example you provided.
After running
>ffprobe -v error -select_streams v:0 -count_packets -show_entries stream=nb_read_packets -of csv=p=0 input.mp4
361
>
I get 361.
Now running the command:
ffprobe -show_format -show_streams -v error -select_streams v:0 -count_packets -show_entries stream=nb_read_packets -of csv=p=0 -of json input.mp4
I get the output
{
"programs": [
],
"streams": [
{
"nb_read_packets": "361",
"disposition": {
"default": 1,
"dub": 0,
"original": 0,
"comment": 0,
"lyrics": 0,
"karaoke": 0,
"forced": 0,
"hearing_impaired": 0,
"visual_impaired": 0,
"clean_effects": 0,
"attached_pic": 0,
"timed_thumbnails": 0
},
"tags": {
"language": "und",
"handler_name": "VideoHandler"
}
}
],
"format": {
"filename": "input.mp4",
"nb_streams": 2,
"nb_programs": 0,
"format_name": "mov,mp4,m4a,3gp,3g2,mj2",
"format_long_name": "QuickTime / MOV",
"start_time": "0.000000",
"duration": "15.045000",
"size": "2299653",
"bit_rate": "1222813",
"probe_score": 100,
"tags": {
"major_brand": "mp42",
"minor_version": "0",
"compatible_brands": "isommp42",
"creation_time": "2013-11-12T06:38:58.000000Z"
}
}
}
That value of 361 is now in key value of nb_read_packets
I tried exactly replicating the command line
out= ffmpeg.probe("input.mp4", cmd='ffprobe', v='error', select_streams='v:0', count_packets=None, show_entries='stream=nb_read_packets', of='csv=p=0')
in ffmpeg-python, but I keep getting a JSONDecodeError:
The value of s from the debugger showed this string value:
s
'361,1,0,0,0,0,0,0,0,0,0,0,0,und,VideoHandler\r\ninput.mp4,2,0,"mov,mp4,m4a,3gp,3g2,mj2",QuickTime / MOV,0.000000,15.045000,2299653,1222813,100,mp42,0,isommp42,2013-11-12T06:38:58.000000Z\r\n'
My only assumption is with the -of or -print_format option, some characters were stripped or modified and that is giving the json decoder issues to decode it.
I finally decided to take out the -of csv=p=0 flag and the following was able to run with no errors
import ffmpeg, json
out = ffmpeg.probe("input.mp4", cmd='ffprobe', v='error', select_streams='v:0', count_packets=None, show_entries='stream=nb_read_packets')
value = out["streams"][0]["nb_read_packets"]
print(value)
361
Side Note
Some of the options that do not have direct parameters following them are defined as None because they do not have any parameters. This has been brought up several times in open issues from the github repo, and I have previously commented on it. It is not stated in the documents, and as far as I know, no PRs are out to update the docs to explicitly state it.
Related
Im going along with the tutorial from:
https://forge.autodesk.com/en/docs/model-derivative/v2/tutorials/prepare-file-for-viewer/
and i got stuck with translating the file to SVF. I've tried data:write, data:create, bucket:update scopes but each time I get:
Token does not have the privilege for this request.
{
"input": {
"urn": "someUrn="
},
"output": {
"formats": [
{
"type": "svf",
"views": [
"2d",
"3d"
]
}
]
}
}
You will need both data:read and (data:write or data:create) (delimited by space) - see doc here:
curl -v 'https://developer.api.autodesk.com/authentication/v1/authenticate'
-X 'POST'
-H 'Content-Type: application/x-www-form-urlencoded'
-d '
client_id=obQDn8P0GanGFQha4ngKKVWcxwyvFAGE&
client_secret=eUruM8HRyc7BAQ1e&
grant_type=client_credentials&
scope=data:read%20data:write
'
And did you select Model Derivative API when creating the Forge app - if not, start over and create the app again:
Thanks,
I've run also with some different translation errors, but it turns out .rfa files are not supported in the Model Derivative API.
But all in all it worked out
First: I startup the sample 'half_plus_two_2_versions' via tensorflow_model_server
Second: I use 'curl -X GET http://localhost:8501/v1/models/half_plus_two_2_versions', but I get the error message: { "error": "Malformed request: GET /v1/models/half_plus_two_2_versions" }
Who can help me, thanks.
I can use the following command get the result:
curl -d '{"instances": [1.0, 2.0, 5.0]}' -X POST http://localhost:8501/v1/models/half_plus_two_2_versions:predict
Result:
{ "predictions": [2.5, 3.0, 4.5 ] }
Thanks.
I already solve this problem.
For tests I used the Google Example of the speech api (https://cloud.google.com/speech-to-text/docs/reference/rest/v1/speech/recognize)
There I tried a .ogg file
This one (https://www.dropbox.com/s/lw66x3g143mtnsl/SpeechToText.ogg?dl=0)
I converted the audio file to 16000Hz
Here is the full request
{
"audio": {
"content": " content "
},
"config": {
"encoding": "OGG_OPUS",
"languageCode": "de-DE",
"sampleRateHertz": 16000
}
}
I converted then the aduio file with an Base64 Encoder (https://www.giftofspeed.com/base64-encoder/) So the content was too long for here.
Now my problem I get just an empty answer. I get the code 200 but nothing else
Thanks for all answers !
The .ogg file URL you referenced was encoded with codec Vorbis not Opus. You can use opus-tools to encode your audio file to an Opus file before you provide it to Google's service
Here's the debugging I used to identify your file as Vorbis:
opusinfo
$ opusinfo SpeechToText.ogg
Processing file "SpeechToText.ogg"...
Use ogginfo for more information on this file.
New logical stream (#1, serial: ffe6c0ca): type Vorbis
Logical stream 1 ended
ffmpeg
$ ffmpeg -i SpeechToText.ogg
ffmpeg version 3.4.2 Copyright (c) 2000-2018 the FFmpeg developers
Input #0, ogg, from 'SpeechToText.ogg':
Duration: 00:00:03.41, start: 0.000000, bitrate: 116 kb/s
Stream #0:0: Audio: vorbis, 16000 Hz, stereo, fltp, 160 kb/s
Metadata:
ENCODER : Lavc58.18.100 libvorbis
My request.sh code:
#!/bin/bash
# Copyright 2017 Google Inc.
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
# http://www.apache.org/licenses/LICENSE-2.0
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# Create a request file with our JSON request in the current directory
FILENAME="request-"`date +"%s".json`
cat <<EOF > $FILENAME
{
"config": {
"encoding":"FLAC",
"sampleRateHertz":16000,
"profanityFilter": true,
"languageCode": "en-US",
"speechContexts": {
"phrases": ['']
},
"maxAlternatives": 1
},
"audio": {
"content":
}
}
EOF
# Update the languageCode parameter if one was supplied
if [ $# -eq 1 ]
then
sed -i 'audio.base64' -e "s/en-US/$1/g" $FILENAME
fi
# Record an audio file, base64 encode it, and update our request object
read -p "Press enter when you're ready to record" rec
if [ -z $rec ]; then
rec --channels=1 --bits=16 --rate=16000 audio.flac trim 0 3
echo \"`base64 audio.flac`\" > audio.base64
sed -i 'audio.base64' -e '/ "content":/r audio.base64' $FILENAME
fi
echo Request "file" $FILENAME created:
head -7 $FILENAME # Don't print the entire file because there's a giant base64 string
echo $'\t"Your base64 string..."\n\x00\x00}\n}'
# Call the speech API (requires an API key)
read -p $'\nPress enter when you\'re ready to call the Speech API' var
if [ -z $var ];
then
echo "Running the following curl command:"
echo "curl -s -X POST -H 'Content-Type: application/json' --data-binary #${FILENAME} https://speech.googleapis.com/v1/speech:recognize?key=API_KEY"
curl -s -X POST -H "Content-Type: application/json" --data-binary #${FILENAME} https://speech.googleapis.com/v1/speech:recognize?key=MY_API_KEY
fi
I am getting the error
{
"error": {
"code": 400,
"message": "Invalid value at 'audio.content' (TYPE_BYTES), Base64 decoding failed for \"ZkxhQwAAACAAA....\"",
"status": "INVALID_ARGUMENT",
"details": [
{
"#type": "type.googleapis.com/google.rpc.BadRequest",
"fieldViolations": [
{
"field": "audio.content",
"description": "Invalid value at 'audio.content' (TYPE_BYTES), Base64 decoding failed for \"Zkxh...
When I executed request.sh bash file, the above error rises.
How can I solve this error? or Does anyone else have a problem like me?
Looks like you'v tried the code from Sara Robinson on Hakernoon too.
I had the same problem and this post from ZodiacLeo123 helped me.
You have to add the the "-w" flag
base64 -w 0 audio.flac
This removes extra new lines.
Provisioning a DSE cluster with the lifecycle manager fails consitently. Master node (also the one OpsCenter is running on) installed correctly. Each one of the other nodes fails the install (also config) task. Have double-checked the SSH credentials and ports. Any ideas on how to investigate further and fix the issue would be great.
Please excuse the length - trying to provide all of the relevant info.
Ubuntu 14.04.4,
JRE: 1.8.0.91,
DSE 5.0.0
job events:
...
"results": [
{
"event-subtype": "start",
"event-type": "milestone",
"message": "job started...",
...
},
{
"event-subtype": "invocation",
"event-type": "shell-command",
"message": "Invoked command: if [ -x $(which yum) ] && [ -f /etc/redhat-release -o -f /etc/SuSE-release ]; then echo -n yum; elif [ -x $(which apt-get) ]; then echo -n apt; fi"
...
},
{
"event-subtype": "uploaded-facts",
"event-type": "milestone",
"message": "Uploaded facts to OpsCenter server",
...
},
{
"event-subtype": "meld-error",
"event-type": "error",
"message": "Unexpected error executing meld",
...
},
{
"event-subtype": "MeldError",
"event-type": "error",
"message": "Meld failed on: name=\"NODE-2\" ssh-management-address=\"<IP>\" node-id=\"<node-id>\" job-id=\"<job-id>\" stdout=\"\r\n\" stderr=\"\"",
...
}
]
opscenterd.log
/var/log/opscenter/opscenterd.log-2016-07-02 16:34:16,848 [opscenterd] INFO: Install job started for node name="NODE-2" ssh-management-address="<IP>" node-id="<node-id>" (async-thread-macro-53)
/var/log/opscenter/opscenterd.log-2016-07-02 16:34:16,850 [opscenterd] INFO: using ssh-private-key (async-thread-macro-53)
/var/log/opscenter/opscenterd.log-2016-07-02 16:34:18,478 [opscenterd] INFO: Received milestone from node name="NODE-2" ssh-management-address="<IP>" node-id="<node-id>" message="Uploaded facts to OpsCenter server" job-id="a630c081-6ac1-4b00-ac08-18fef320e0d5" (MainThread)
/var/log/opscenter/opscenterd.log:2016-07-02 16:34:18,675 [opscenterd] ERROR: Received error from node event-subtype="meld-error" job-id="a630c081-6ac1-4b00-ac08-18fef320e0d5" name="NODE-2" traceback="Traceback (most recent call last):
/var/log/opscenter/opscenterd.log: File \"meld.py\", line 3313, in run
/var/log/opscenter/opscenterd.log- rc = engine.go()
/var/log/opscenter/opscenterd.log: File \"meld.py\", line 2991, in go
/var/log/opscenter/opscenterd.log- self.file_manager.get_config_files()
/var/log/opscenter/opscenterd.log: File \"meld.py\", line 1280, in get_config_files
/var/log/opscenter/opscenterd.log- {\"accept\": \"application/json\"})
/var/log/opscenter/opscenterd.log: File \"meld.py\", line 598, in get
/var/log/opscenter/opscenterd.log- return json.loads(response.read())
/var/log/opscenter/opscenterd.log- File \"/usr/lib/python2.7/socket.py\", line 351, in read
/var/log/opscenter/opscenterd.log- data = self._sock.recv(rbufsize)
/var/log/opscenter/opscenterd.log- File \"/usr/lib/python2.7/httplib.py\", line 549, in read
/var/log/opscenter/opscenterd.log- return self._read_chunked(amt)
/var/log/opscenter/opscenterd.log- File \"/usr/lib/python2.7/httplib.py\", line 609, in _read_chunked
/var/log/opscenter/opscenterd.log- value.append(self._safe_read(amt))
/var/log/opscenter/opscenterd.log- File \"/usr/lib/python2.7/httplib.py\", line 666, in _safe_read
/var/log/opscenter/opscenterd.log- raise IncompleteRead(''.join(s), amt)
/var/log/opscenter/opscenterd.log:IncompleteRead: IncompleteRead(4153 bytes read, 4039 more expected)" ssh-management-address="<IP>" node-id="<node-id>" event-type="error" message="Unexpected error executing meld" (MainThread)
/var/log/opscenter/opscenterd.log-2016-07-02 16:34:18,892 [opscenterd] ERROR: Install job a630c081-6ac1-4b00-ac08-18fef320e0d5 failed! (async-thread-macro-54)
/var/log/opscenter/opscenterd.log:2016-07-02 16:34:19,105 [opscenterd] ERROR: Meld failed on: name="NODE-2" ssh-management-address="<IP>" node-id="<node-id>" job-id="a630c081-6ac1-4b00-ac08-18fef320e0d5" stdout="
/var/log/opscenter/opscenterd.log-" stderr="" (async-thread-macro-53)
Thank you
EDIT: Captured the HTTP traffic between NODE2 and master. The error occurs while transferring config files. One of them is not transferred completely for some reason. The json looks resonable until some gibberish appears.
{"filename": "dse.yaml", "contents": {"internode_messaging_options": {"client_worker_threads": 16, "port": 8609, "server_worker_threads": 16, "server_acceptor_thread
Yvatv+~UK{.kMI4^QOrqQTDX_3"DPm,v!"H&M$!1M7
LRYCs{l>-df;cj
W6C9dq
The config files are valid and do work on the master node. Only the replication fails.
OpsCenter LCM developer here. Your issue is caused by OPSC-8851 in the LCM known issues list: http://docs.datastax.com/en/opscenter/6.0/opsc/release_notes/opscReleaseNotes600.html
This is only triggered under certain network conditions and was discovered too close to release to get fixed in 6.0.0. It's a high priority though, and will be fixed in a subsequent release soon. Unfortunately, I don't think there's anything you can do to work around this in the field. If you're a DataStax customer, you could contact support and potentially get a patch now to workaround the issue... otherwise the only thing I can suggest is to watch the upcoming release notes.
Edit: I should also note that in our tests the issue is intermittent. LCM is designed so you can rerun failed jobs safely (aka it's idempotent) so in all but the most extreme cases you can also work around this just by rerunning your job.
You can specify the private IP for Listen Address and 0.0.0.0 for broadcast address and LCM should be able to provision appropriately.