Some websites provide pdf files for viewing but I can't download such pdf files with wget.
Calling the website in question from my browser views the pdf:
https://www.lokalmatador.de/epaper/ausgabe/gemeinderundschau-muehlhausen-14-2021/
But using the following code I only get a pdf file with 0 lenght.
wget --content-disposition -nd https://www.lokalmatador.de/epaper/ausgabe/gemeinderundschau-muehlhausen-14-2021/
I tried some combinations with saving and loading cookies and referer but nothing works.
At this point I'm just curious what is happening and why wget is not fetching anything except maybe an empty index.html.
When I was looking at server response, it was saying the content length was 0.
--2021-04-17 14:59:35-- https://www.lokalmatador.de/epaper/ausgabe/gemeinderundschau-muehlhausen-14-2021/
Resolving www.lokalmatador.de (www.lokalmatador.de)... 37.202.6.70
Connecting to www.lokalmatador.de (www.lokalmatador.de)|37.202.6.70|:443... connected.
HTTP request sent, awaiting response...
HTTP/1.1 200 OK
Date: Sat, 17 Apr 2021 13:59:36 GMT
Server: Apache
Set-Cookie: fe_typo_user=477e8a1d2b3dd74bc5b6b408a6d74edd; expires=Mon, 17-May-2021 13:59:36 GMT; Max-Age=2592000; path=/; domain=.lokalmatador.de; httponly; samesite=lax
Upgrade: h2,h2c
Connection: Upgrade, Keep-Alive
Content-Length: Array
Cache-Control: max-age=2592000
Expires: Mon, 17 May 2021 13:59:36 GMT
X-UA-Compatible: IE=edge
X-Content-Type-Options: nosniff
Keep-Alive: timeout=5, max=100
Content-Type: application/pdf
Length: 0 [application/pdf]
Remote file exists but does not contain any link -- not retrieving.
So looked at the manual:
https://www.gnu.org/software/wget/manual/html_node/HTTP-Options.html
And there is a command just exactly for this:
‘--ignore-length’
Unfortunately, some HTTP servers (CGI programs, to be more precise) send out bogus Content-Length headers, which makes Wget go wild, as it thinks not all the document was retrieved. You can spot this syndrome if Wget retries getting the same document again and again, each time claiming that the (otherwise normal) connection has closed on the very same byte.
With this option, Wget will ignore the Content-Length header—as if it never existed.
Then the wget command started working as expected:
wget --ignore-length -O epaper.pdf https://www.lokalmatador.de/epaper/ausgabe/gemeinderundschau-muehlhausen-14-2021
Here is output which I'm seeing with the ignore length:
--2021-04-17 14:56:19-- https://www.lokalmatador.de/epaper/ausgabe/gemeinderundschau-muehlhausen-14-2021
Resolving www.lokalmatador.de (www.lokalmatador.de)... 37.202.6.70
Connecting to www.lokalmatador.de (www.lokalmatador.de)|37.202.6.70|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: ignored [application/pdf]
Saving to: ‘epaper.pdf’
epaper.pdf [ <=> ] 4.39M 1.23MB/s in 3.6s
2021-04-17 14:56:23 (1.21 MB/s) - ‘epaper.pdf’ saved [4601842]
I'm trying to send a voice message from a Telegram BOT, without success.
As a proof of concept I'm trying to do that by using curl:
#!/bin/bash
TOKEN=$(cat .token)
CHAT=$(cat .chat)
URL="http://server2.mbrt.it:8080/static/foo.ogg"
curl "https://api.telegram.org/bot$TOKEN/sendVoice?chat_id=$CHAT&voice=$URL"
But I keep getting:
{"ok":false,"error_code":400,"description":"Bad Request: wrong file identifier/HTTP URL specified"}
I'm sure that both the token and the chat id are correct, because I'm able to send audio with the sendAudio method (sending an mp3 file). The URL I'm using is public, a wget from any PC will download the file.
The HTTP headers are also correct AFAIK (note the Content-Type header):
$ curl -v http://server2.mbrt.it:8080/static/foo.ogg >/dev/null
< HTTP/1.1 200 OK
< Server: nginx/1.10.3
< Date: Fri, 09 Jun 2017 14:14:11 GMT
< Content-Type: audio/ogg
< Content-Length: 5881
< Last-Modified: Thu, 08 Jun 2017 23:23:21 GMT
< Connection: keep-alive
< ETag: "5939dc69-16f9"
< Accept-Ranges: bytes
My guess was that my encoding was wrong in some way:
$ file foo.ogg
foo.ogg: Ogg data, Opus audio,
But I tried to encode it either with ffmpeg, opusenc and oggenc. In all cases I get the same error when I send it.
I have no idea of what I'm doing wrong.
I tried it on my own VPS, seems Telegram only accept port 80 for HTTP, and port 443 for HTTPS :(
You can download it, and use curl -F "voice=#foo.ogg" to upload it yourself.
My website is visible (http://www.nowis-informatique.fr) but when I try a :
curl -I http://www.nowis-informatique.fr/assets/front/img/slider/2/-tincelles-de-couleurs_480x480.png
My header is :
HTTP/1.1 404 Not Found
Date: Fri, 16 Jan 2015 13:14:41 GMT
Server: Apache
Connection: close
Content-Type: text/html; charset=iso-8859-1
But when i try with arg -4 on curl, it's work.
Reallity problem :
When sharing a facebook page (if I copy the URL in a new Facebook status, for example), the image and description generated by Facebook is not at all that of my site, but homepage of my server.
example: www.nowis-informatique.fr
Accessible without worries from my browser. When I put the URL into a facebook status he me out of my server page "87.106.242.7" ...
On this server I have several field and it does the same for them. I want to say that everything worked on 09.22.2014, the date of my last Facebook publication.
I looked on the developers.facebook.com/tools/debug/ software The return code is 206, and the information is that of my "87.106.242.7" obviously ...
I do not know how strictly fix the problem ...
Thank you all
Apache v2.2
I'm failing a PCI compliance scan because my Railo server is revealing the path to the document web root in an "exception-message" header when a missing page is requested. I tried using both the built-in Railo 404 template and my own custom 404 template to no avail. Is there anyway to get rid of this header from the reponse?
$ curl -I http://mydomain.com/this-page-does-not-exist.html
HTTP/1.1 200 OK
Date: Wed, 08 Jan 2014 22:46:20 GMT
Server: Apache-Coyote/1.1
exception-message: Page /this-page-does-not-exist.html [/var/www/html/this-page-does-not-exist.html] not found
Content-Type: text/html;charset=UTF-8
Content-Length: 44
Set-Cookie: CFID=31254774-4b81-470d-b0da-dfadd4585ce0;Path=/;Expires=Fri, 08-Jan-2044 06:37:50 GMT
Set-Cookie: CFTOKEN=0;Path=/;Expires=Fri, 08-Jan-2044 06:37:50 GMT
Connection: close
Update: I was able to fix this problem by overwriting the header.
I created a custom 404 template and then set the Missing Template Error (404) option to point at it in the Railo administrator. Then I added this line of code to the top of the page which seems to overwrite the header with a blank string.
<cfset getPageContext().getResponse().setHeader("exception-message","")>
Note: Using the tag <cfheader> to do the same thing does not work. I'm not sure why but the Java route seems to do the trick.
I need to either find a file in which the version is encoded or a way of polling it across the web so it reveals its version. The server is running at a host who will not provide me command line access, although I can browse the install location via FTP.
I have tried HEAD and do not get a version number reported.
If I try a missing page to get a 404 it is intercepted, and a stock page is returned which has no server information on it. I guess that points to the server being hardened.
Still no closer...
I put a PHP file up as suggested, but I can't browse to it and can't quite figure out the URL path that would load it. In any case I am getting plenty of access denied messages and the same stock 404 page. I am taking some comfort from knowing that the server is quite robustly protected.
The method
Connect to port 80 on the host and send it
HEAD / HTTP/1.0
This needs to be followed by carriage-return + line-feed twice
You'll get back something like this
HTTP/1.1 200 OK
Date: Fri, 03 Oct 2008 12:39:43 GMT
Server: Apache/2.2.9 (Ubuntu) DAV/2 SVN/1.5.0 PHP/5.2.6-1ubuntu4 with Suhosin-Patch mod_perl/2.0.4 Perl/v5.10.0
Last-Modified: Thu, 02 Aug 2007 20:50:09 GMT
ETag: "438118-197-436bd96872240"
Accept-Ranges: bytes
Content-Length: 407
Connection: close
Content-Type: text/html; charset=UTF-8
You can then extract the apache version from the Server: header
Typical tools you can use
You could use the HEAD utility which comes with a full install of Perl's LWP library, e.g.
HEAD http://your.webserver.com/
Or, use the curl utility, e.g.
curl --head http://your.webserver.com/
You could also use a browser extension which lets you view server headers, such as Live HTTP Headers or Firebug for Firefox, or Fiddler for IE
Stuck with Windows?
Finally. if you're on Windows, and have nothing else at your disposal, open a command prompt (Start Menu->Run, type "cmd" and press return), and then type this
telnet your.webserver.com 80
Then type (carefully, your characters won't be echoed back)
HEAD / HTTP/1.0
Press return twice and you'll see the server headers.
Other methods
As mentioned by cfeduke and Veynom, the server may be set to return limited information in the Server: header. Try and upload a PHP script to your host with this in it
<?php phpinfo() ?>
Request the page with a web browser and you should see the Apache version reported there.
You could also try and use PHPShell to have a poke around, try a command like
/usr/sbin/apache2 -V
httpd -v will give you the version of Apache running on your server (if you have SSH/shell access).
The output should be something like this:
Server version: Apache/2.2.3
Server built: Oct 20 2011 17:00:12
As has been suggested you can also do apachectl -v which will give you the same output, but will be supported by more flavours of Linux.
Warning, some Apache servers do not always send their version number when using HEAD, like in this case:
HTTP/1.1 200 OK
Date: Fri, 03 Oct 2008 13:09:45 GMT
Server: Apache
X-Powered-By: PHP/5.2.6RC4-pl0-gentoo
Set-Cookie: PHPSESSID=a97a60f86539b5502ad1109f6759585c; path=/
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
Connection: close
Content-Type: text/html
Connection to host lost.
If PHP is installed then indeed, just use the php info command:
<?php phpinfo(); ?>
Rarely, a hardened HTTP server is configured to give no server information or misleading server information. In those scenarios if the server has PHP enabled you can add:
<?php phpinfo(); ?>
in a file and browse to it and look for the
_SERVER["SERVER_SOFTWARE"]
entry. This is susceptible to the same hardening lack of information/misleading though I would imagine that it's not altered often, because this method first requires access to the machine to create the PHP file.
The level of version information given out by an Apache server can be configured by the ServerTokens setting in its configuration.
I believe there is also a setting that controls whether the version appears in server error pages, although I can't remember what it is off the top of my head. If you don't have direct access to the server, and the server administrator is competent and doesn't want you to know the version they're running... I think you may be SOL.
Telnet to the host at port 80.
Type:
get / http1.1
::enter::
::enter::
It is kind of an HTTP request, but it's not valid so the 500 error it gives you will probably give you the information you want. The blank lines at the end are important otherwise it will just seem to hang.
If they have error pages enabled, you can go to a non-existent page and look at the bottom of the 404 page.
Your best option is through PHP:
All version requests from the client side cannot be trusted since your Apache could be configured with ServerTokens Prod and ServerSignature Off. See: http://www.petefreitag.com/item/419.cfm
In the default installation, call a page that doesn't exist and you get an error with the version at the end:
Object not found!
The requested URL was not found on this server. If you entered the URL manually please
check your spelling and try again.
If you think this is a server error, please contact the webmaster.
Error 404
localhost
10/03/08 14:41:45
Apache/2.2.8 (Win32) DAV/2 mod_ssl/2.2.8 OpenSSL/0.9.8g mod_autoindex_color PHP/5.2.5
Simply use something like the following - the string should be there already:
<?php
if(isset($_SERVER['SERVER_SOFTWARE'])){
echo $_SERVER['SERVER_SOFTWARE'];
}
?>
Use this PHP script:
$version = apache_get_version();
echo "$version\n";
Se apache_get_version.