I'm twinkering around in a hobby project and would like to know, is it possible to host a Wordpress backend as my headless CMS and my frontend Vue Vite on the same server?
I would like to achieve this, because when I have Wordpress on another server completely, it takes about 800-1000ms for the API to load the content (Rest Api cache enabled with plugin), even when its like 6 posts each containing 1 midsize (around 1000x1000) image and 100-200 words?
Or should I look for another solution, eg. completely move to another CMS like buttercms?
If the latter, can I host that CMS on the same server to cut down loading/response times of rest api requests?
Thank you for your time,
Have a nice one!
Related
I've experimented with some crawlers to pull web data from within a Python environment on my local machine. Ideally, I'd like to host a website that can initiate crawlers to aggregate content and display that on the site.
My question is, is it possible to do this from a web environment and not my local machine?
Sure there are many services that are doing the same task you wanted.
scrapingHub is the best example you can get. https://scrapinghub.com/
You can deploy your spiders in there and run it periodically(paid service). Deploy and call spider via scrapingHub API form your website and use the spider output in your host website.
Also, you can achieve the same idea in your server and website via API call.
I have been developing a crawling script for a number of news websites and using Scrapy to handle the logic.
When I run my script on an Ubuntu web server (Digital Ocean, if that helps), a lot of the websites that return 200 on my local machine turn out to be 417 instead.
I was wondering how I should fix this, if it is a problem at all? I'm actually not quite sure if it is affecting the final output, but it seems like it has been.
Some of my own research has turned up:
http://www.checkupdown.com/status/E417.html . I've tried adding an Expect header to my requests, which hasn't worked
I've heard that it might be a problem with HTTP 1.1 vs 1.0? EDIT: Nope. Scrapy's HTTPDownloaderHandler automatically chooses 1.1 if it is available
417 is the error a web server gives you when your client says it expects content-types a,b,c, but the content that the server could deliver doesn't match any of these types.
This looks like a scrapy bug or, more likely, misconfiguration.
It seems either your public ip address was already banned or was banned while you scraped by the web server of the page you want to scrape. For the first situation you can reboot your instance to get a new public ip (at least this works on Amazon). For the second scenario, here are some tips from the official documentation to avoid this situation:
rotate your user agent from a pool of well-known ones from browsers
(google around to get a list of them)
disable cookies (see COOKIES_ENABLED) as some sites may use cookies to spot bot behaviour
use download delays (2 or higher). See DOWNLOAD_DELAY setting.
if possible, use Google cache to fetch pages, instead of hitting the
sites directly
use a pool of rotating IPs. For example, the free Tor
project or paid services like ProxyMesh
use a highly distributed downloader that circumvents bans internally, so you can just focus on parsing clean pages. One example of such downloaders is Crawlera
Additionally, you can reduce concurrent requests settings in your spider, that worked once for me.
We have a DotNetNuke installation that we have on a dev environment and we are setting it up on our live server now. Everything looks great and is working just fine that regard. We were wondering if it was possible that once a page has been updated or created on our dev server and we're happy with it, is there a simple and easy way to push that page to the live server?
Dev has it's own database.
Live has a different database to itself.
We have seen some programs like SQL Sync but we do not want to push all pages from dev to live because there might still be some pages being worked on when another one finishes so we must have a way to push individual pages instead of everything.
Thank you so much for any help!
you can use Exporting And Importing feature in dotnetnuke it allows you to export a single page or whole web site you can follow this links for helo
Page import export
Great article to look for
So first of, hi,
I have probably a simple issue but I'm not expert enough to figure out what the problem is.
I just deployed a zend framework application on a debian server. I have dotster as registrar and another provider for the dedicated server. The problem is about the domain mapping and the urls when navigating from page to page within the app.
Basically when I use the server ip, like 91.121.121.171/public/ to reach the app, the url works just fine and displays well in the address bar like 91.121.121.171/public/directory/dir-comp for example. But when I use the url www.ceramstar.org this stays fixed. Did I mess up with dotster config or Apache rewrite rule?
I must say I don't really get how to configure things properly on the dotster side...
Thanks a lot anyway
I am suggested by a friend to use nginx for my site http://www.imgzzz.com . its an image hosting site with loads of traffic.
Currently im on a vps ..CentOS 5.4 x64 , Apache
Most views are on image pages. So far to decrease the load on server i have done cache of almost all data like user details, image name, path, category details etc.
Still i have to go with about 3 sql queries every page/view.
Addition of Views
Displaying Views
Addition of user ad views with respect to their ads shown
Considering the traffic from social media sites like digg, stumbleupon. per second online user peaks upto 1500-2500. So i guess you can get the idea of php queries per second.. Sometimes it causes the server to lag.
The rest of the stuff on image pages is static. So now do you guys suggest nginx or any other better option for my server?
Thanks in advance :)
Edit : This is a custom system not any cms
I would recommend using nginx as your static file server. I run nginx for this and it works great, and I could vouch for alot of other people I know that uses it. It's fast and reliable.