Program that can extract something from a website - extractor

I`m sorry if this is the not right place to ask. If not please delete this.
I`m looking for someone that can help me with a very simple program that can do this:
Login into accounts on an website and extract there a number from every account and and save the details ( account number - the number that must be saved ) on an txt file.
I need this for my job. This program will save me a`lot of time. this is the purpose.
If anybody can help me please let me know. thank you very much.

Fortunately for you, there is ample tooling available.
I will provide you some insight into related tools in the Python ecosystem, since that is what I am most familiar with and also an easy languages for beginners to work with.
If what you are extracting/scraping is relatively simple and doesn't require complex UI interaction with website elements, I would recommend requests Sessions to retain cookies and additional information for use in a series of authenticated requests, and bs4 to parse document trees in order to extract the data you are interested in.
For more complex interaction, you will need to look towards browser automation and possibly more advanced scraping frameworks. Hopefully I've given you enough keywords to go far in Google.
Of course, you'll need to first learn the basics of programming and how web data is structured if you want to write scripts like this. That is left to you to learn and absorb.

Related

GET, then manipulate that info (while loop and if statements), then PUT

I feel like this is a simple enough thing to do and can't believe how hard of a time I'm having finding the example I need so alas, human help please! You failed me Google :)
FYI I've written a bit of code in my life (Java, C, ASM, PHP), new to APIs.
As the title lays bare, I want to retrieve info (tickets out of our ticketing system meeting certain conditions) via a GET request, loop through each ticket, and update each one (PUT) based on certain conditions.
If there's a tutorial you know of that covers that, please point me to it! If not, if you could please fill in some of these holes?
I wound up in Postman, is this an appropriate environment to accomplish such? Of course you can make individual API calls here but I got lost trying to string the aforementioned sequence together. Is it possible via "Create API?" This makes it sound like I'm creating my own API server for other users to access data from. If it is possible via "Create API," where's the dang run code button?! If not Postman, what tool should I be using to write code in? And again, if you have any code samples/videos closely resembling said scenario for that tool.
Thank you for any assistance you can offer.
Cheers,
Jay
You want to interact with your ticketing system: read existing tickets, and update them. Correct?
To do this, your TICKETING SYSTEM must have an "api" (possibly - but not necessarily - a REST API).
If it does, you might be in luck. But you need to learn the API first, possibly from product documentation. If it doesn't, there's probably not much you can do except to interact with the system as a "normal user".
If it has an API, and if it's a relatively simple API (like REST), you can use the API manually (e.g. through POSTMAN), or you can use it programmatically (using Python, C#, Java or just about any programming language you feel comfortable with).
In summary, you need to:
Determine if your ticketing system provides any kind of web api
If so, get the API details
If at that point you want proceed, determine what programming language or framework you'd like to use.
Hopefully that gives you a bit of "direction".

A Conceptual Understanding of APIs

I have been learning coding for about a month now. I have some good experience with Python, and additionally I have completed this web development course on Udacity.
Now, I have a good foundation for programming, but one thing that confused me a lot is how to interact with various websites and APIs. The course I did briefly touched on this in terms of XML and JSON files and how some webpages offer their pages in these formats for easier reading by other machines.
But there are still a bunch of tasks which I have no idea how to approach whatsoever, but want to eventually do. I have constructed some hypothetical examples for the purpose of this question. I will post my current rough understanding of how I would do them below each one, and would appreciate feedback (on the API interaction, not on the front-end or on any back-end algorithms/AI/parsing):
Creating a phone application (disregarding the front-end part) which can then communicate with and perform rudimentary tasks on my computer.
I have no idea how to do this, and my guess would be that I would have to look into some external application/API meant for this process and implement this on both-ends of the system.
Being able to write a bot which goes on to a game website and controls the object via script. (e.g going onto a pacman game website written in flash and automatically controlling the character to avoid the ghosts)
I don't even know if this is possible, or how browser flash games interact handle the user-server interaction. Would I have to post some data via HTTP manually in the same way that playing in the keyboard would do? Or is everything done client side, in which case how would I fake user input? How would I get information on the ghost's position to work the AI?
Creating a mobile app for my school by allowing users to put their username and password into the app and then having the app automatically log in to the school and fetch certain data (e.g timetable) and return back in a readable form.
I'm guessing that I would take the input from the user on my mobile-app, and then navigate to the school's login page and POST this data in the relevant forms to log in. And then that I would (somehow, not sure), navigate to the timetable URL through my code while still managing to stay logged in, and then parse the html there?
I would appreciate some detail on how these kind of things are done, preferably with reference to these examples, so that I can get a better conceptual understanding.
Thanks!
Note: I have asked all those various questions mostly rhetorically, just so that those reading can get a better understanding of what my current programming level and understanding of APIs is at. I do not expect nor require specific answers for each and every question (so I hope this doesn't get flagged as being too vague or requiring too much detail!), I just would appreciate some responses telling me roughly how each of these APIs work approximately and how I would even start at looking at how to do these things.
You asked too many questions and honestly speaking I am not able to read and grasp entire text posted by you.
So, I am focusing only the title of your question:
"A conceptual understanding of API"
API (Application Programming Interface) means a set of functions which you can directly use by simply passing parameters to them.
Actually, in application development there are many common functions which every application programmer have to use. So, instead of coding them every time by every programmer, they are already coded in functions which you can use simply by passing parameters to them (if they need any external parameter).
Example:
I am offering you a maths API, set of functions {add, sub, mul, div}. You can pass two numbers to any of these four functions and get desired result instead of coding every time for ever operation like add, sub, mul and div.
Hope it helps...

Amazon search bar api Product advertising

I am trying to place amazon search bar for books category in my website, where user searches for a book and is redirected to amazon website. My URL should be tagged with my associate ID so that i can earn some money.
The problem is i am unable to find any procedure to create such search bar. I have browsed through Product advertising API section, but it is very confusing.
I want exactly like this: http://amasearchbar.com/demo-blog/
Can someone help me how to make such autocomplete search bar, or provide me directly the code for it.
Any help is appreciated.
Amazon isn't likely to give instructions for how to create something like that - but they do give you the tools necessary to figure it out yourself if you're willing to learn their API.
Without more information, it's hard for anyone here to "help" you either.
That link you gave is to a WP plugin. Are you using WP? If so, your easiest bet is to buy the plugin and use that (I am not endorsing that plugin specifically, I have no experience with it).
If you give more specifics on what language you're wanting to use to create the search bar and its interaction with the API, and specific problems you run into while working on it, this community will be much more likely to help you. Questions to point you to a free source for the finished code of your project are not likely to get very far though.
Based on your previous questions, it seems likely that you are using Wordpress. If that's the case, and you don't know how to write the scripts to interact with the API then the easiest answer to your question by far is to either buy a plugin (perhaps the one you linked) or hire someone to write one. If you'd like to learn, there are a lot of resources online to help you start learning to write WP plugins.

Creating dynamic templates with GAS

We've just decided to purchase Google Apps for Business for a couple of users and are now running pilot for migration.
We have a certain issue, which we would like to see if it can be solved with Google Apps Script.
Suppose that the following situation applies:
At this moment we're in a domain and we have certain user templates set-up in documents such as a document for faxing and a document for memo's. These templates have application-logic behind them so the template gets filled in values for the users name, and branch office, ...
What we would like to do is replicate this behaviour in Google Drive, but I'm not sure what the best way is to implement such a feature.
Do we create a spreadsheet and in the script editor we write the full template from scratch? Or is it possible to have a template on the google apps account which we then can transform on open?
I hope I'm a bit clear of what we would like to achieve, but in case I'm not; do feel free to ask me more questions.
I hope to hear from you all!
Kind Regards
Your question relates more to a global appreciation of Google Apps than a script question... I'm not sure I should be answering here...
As a personal opinion I guess documents would be the best tool to get what you want. You'll have to create a couple of templates with personalization fields that a script would fill in with the 'logged user' data.
The document and doclist api are quite powerful and could also classify the created docs in folders and manage how they are eventually shared among users.
Depending on your abilities in javascript coding it can be anything between quite easy and really hard to build up ... :-)

Google Chart APIs for plotting data in local machine

Preface:
I am a system programmer (who has just started his career as a S/W Engineer), so not very good # web scripting languages, though I have just started learning them.
Problem Synopsis:
I want to write an app that keeps track of what I am doing and records it allowing me to analyse my time spending pattern and could help me analyse whenever I want.
Problem Description:
My plan is to write an app that sits in the background and keeps track of active window on my desktop (every second) and stores this data in a SQLite database. But to be more appealing (As I want to share this app with others), I want to have a feature where the user can analyse data recorded between any period of his interest. For this I want the user to be able to generate charts and graphs using the recorded data.
For this I thought of using browser for UI and Google Visualization APIs for plotting work. So, is it possible to use Google Visualization APIs to plot local data? if so, plz guide me on how to continue... (As told before, I am a system programmer, C programmer to be specific, who has just started learning web scripting in free time)
Reasons for these decisions:
(1) App that records what I am doing will be in C/C++ - B'coz I am system programmer and am very comfortable with them. And can get it done easily and quickly.
(2) SQLite - Very small and can easily be embedded in my app, and is Open Source. And I think many web scripting langs like PHP, python have interfaces to access SQLite DB.
(3) Browser for UI - Hope it will be easy for user to use browser, and I will not have much to do regarding UI. As main UI will be browser and Google Visualization APIs will do plotting. All I might have to do is write few lines of script (Am I right here on the last point???)
Comment on my design decision and any tutorials(or pointers) which teaches me on how to do this will be highly appreciated...
Thank You
MicroKernel :)
PS: Idea inspired by Nathan Baulch's reply to https://stackoverflow.com/questions/161590/how-do-you-track-your-time
#Nathan Baulch, Thank you so much for such a brilliant idea. \m/
I would embed browser to app (you want to write in C#)
and use jquery plotting as chart. You will find more info here: http://www.flotcharts.org/