I have a website and for internal page I have take keyword as hyderabad and I kept this keyword hyderabad in cookies so that when we surf the subcategories we would get that page from the cookies... So my doubt is when crawler visits the subcategory page then does it identifies cookies and can identify the keyword....if yes how...
It is always best to assume they're not using cookies. Plus, real users themselves can disable cookie if they don't find your site to be trusted enough.
It would probably be better to use the keyword as a SESSION variable instead of a cookie.
Related
I am doing a Fullstack Web Project at the moment and am now looking into API security. I have implemented a user login which then allows the client to access most routes of my API. But there are unprotected routes as well which can and must be accessed without a login (e.g. sign up, login, ...).
One route for example returns whether a specific username or e-mail-adress is already taken by another user and is used during the sign up process. For someone with bad intentions this public route would be a great help to figure out which e-mail-addresses or usernames to try to hack or guess the password for because they can avoid credentials that aren‘t in use.
I know that this functionality is used very often by big companies to show the user whether he can use his desired username or not. How are they making sure there‘s no abuse? How can I protect my API and users from that?
I would appreciate any kind of help. Thanks a lot!
To limit API abuse rate limiting is a good practice. One way is to specify a maximum amount of API requests per IP address and after that ignore the requests.
A good article describing solutions to do so and also showing examples for an implementation in NodeJS can be found here.
Thanks to Gilbert Le Blanc for the help on this one.
We have a site that uses a "one-time" login process for password resets which are not initiated by the user themselves. (for instance, a password reset that is initiated by an admin or another employee) A URL is sent to the user via email which can then be used to reset their password. The URL can only be visited one time. (there's more to this for security-sake but I'll keep it simple) Recently, some users have complained that when they visit the link, it has already expired. The end result is that they can't reset their passwords using this feature. We discovered that the users in question have a spam filter or "link checker" in their environment that they do not have access to. This device visits the one-time link before the user is able to, to make sure its safe.
I'm trying to solve this issue and was wondering if there's a way I can detect these type of devices on the web server when the request is made? When the spam filter visits the link, is there something in the http request that would stand apart from a regular browser? Maybe they all use a specific custom HTTP header? Or maybe there's a regex I could use on the user agent? I haven't been able to catch one of these yet, so I'm not sure what the request looks like coming from a spam filter.
Anyone know of a way to detect spam filters of any vendor by looking at the http requests? I know it's a long shot but maybe they all use a specific header for reasons such as this?
I got approval to modify the design to remove the one-time aspect of the URL. This solves the issue and saves me the headache. Thanks for the suggestion, #PeeHaa
Brief
I have been trying to write rules to protect a website from DoS attacks and the website have only one page /index.php. However, each user is identified by an unique token like this /index.php?a=abcdef. This page sends a lot of ajax calls that gets and posts information, periodically, as long as the user is active in the page.
Problem
During a DoS attack, I need to block only the specific user using the token ?a=abcdef, which is dynamic. And the actual length of the token is around 45 characters, consisting of [a-zA-Z0-9]. The website is visited by either a single user or user group (more than 10 users) from an IP address. The problem is, during a DoS attack, when blocking the user, I need to take the token into consideration. However, as far as I understand, unless I use a dynamic variable like, tx.dos_counter_%{ARGS_GET}=+1, I may not be able to identify an user uniquely. But when I reference the variable like this,%{tx.dos_counter_%{ARGS_GET}} to retrieve the value of the dynamic variable, it returns null.
Questions
Is it possible to use dynamic variables, as mentioned above, in ModSecurity?
Is there any other way to handle this problem?
Miscellaneous
This website runs on LAMP stack (Ubuntu) with ModSecurity.
Please help me solve this issue.
Many thanks in advance.
Thanks,
Daniel
I'd like to create a piece of code that can be embedded on many different websites (widget).
Is there any way that my code can identify a user without them logging in? I.e, can I use any of the established identity mechanisms floating around the web to reliably identify them across instances of this widget?
I don't need to (nor should I be able to) tap into any information about this user, just identify them.
The websites will be heterogeneous; there's no guarantee that they will have any common aspects, so the widget code needs to be entirely self contained.
What you want to do is what cookies were invented for. But browsers have gotten wise to people being tracked without their permission, and now limit 3rd party cookies.
The Electronic Frontier Foundation recently put up a proof of concept for uniquely identifying a visitor based on attributes of their browser. It's uses things things like:
User-Agent string
http-accept values
timezone
screen resolution and color depth
the installed plugins
if cookies are enabled
It's not guaranteed to be unique, but my browser certainly is, and it will get you on your way to doing the bad things that people don't like.
OpenID is sort of a SSO for the whole internet, yet they still have to sign in to OpenID. Other than that, I can't think of a solution.
I would suggest Open-ID rather than some workaround like this, but if you don't like that solution you might consider something like this:
You can use a cookie from a single domain, then use that domain to redirect to the correct site adding user-id as a parameter or part of the URL-path
For instance a link to add a personal widget on the foo.com -site, could look something like:
http://bar.com/addwidget1?backtoo=http://foo.com/main/
bar.com would own the cookie, change the user preferences and then add user-id to the back-link before re-directing:
foo.com/user-id/
Issues with this approach includes
You need to rewrite every page dynamicly with the user-id.
It's a bit clumsy I think
You can't fully take advantage of web-caches around the net.
The user might loose their cookie.
Benefits
No login
Since you redirect a lot you get stats on the users movement across your sites.
Sounds like you want to implement a Single Sign On framework. Basically when you first see the user, if you don't know them, you redirect them to the single sign on server. Wich authenticates them and redirects them back to you with a authentication token. You verify the authentication token with a web service call to the SSO server. Ff it is valid then you mark that user as signed on.
EDIT
So thinking about it more and reading tovare's answer and your comments. Why not create some javascript code that works like an google ads? You put the javascript on the page and it does an a request to your central tracking server using a dynamic iframe.
Have your tracking server return an image tag with src of the unique id (its own session id) embedded.
<img src=contentserver.com/track.php?id=12345668>
The content server has a server side script (track.php above) that maps its local session id to the unique id received from the tracking server.
The unique id stays the same across all sites.
Edit2
Instead of using an image, use the javascript trick. The content server just requests a javascript file from the tracking server. but the file is a dynamic one generated on the server side. it returns a generated javascript function called unique_id() it returns the unique id from the tracking server. Call the track.php using ajax to determine if this is a unique user.
Use OpenID, or a simplified variant, with your own site as the identity provider. Redirect the user to your identifier site which sets or checks a cookie, then redirect the user back with the user's identity, which was indicated by the cookie, added as a URL argument.
Your identifier site can be an OpenID identity provider which doesn't require any user interaction to authenticate. The sites which receive this identity are probably not OpenID consumers, since they don't offer the user a choice of providers. You can probably do away with some of the signing required by OpenID if your cookie and identifier are signed.
Facebook provides something similar; a site can find the Facebook identity of a user (assuming the user has one) without any action on the user's part.
I've been thinking about this quite a bit lately, and I would like some feedback from this wonderful community. Is it safe to assume that a user wants to be remembered when they login? And if they are using a public computer, is it safe to assume that they are smart enough to logout before leaving?
I don't really think it's safe to assume anything about the end-user.
Plus, it's easier for a user to just close the browser rather than to log out of every website, so providing a remember me checkbox defaulted to false is user friendly and less hassle for them over-all.
No. In this age of too-little privacy, any potential hole for a breach should be closed. Sites should be getting more secure, not less.
No, it is best to assume nothing, especially something like remembering the user.
What if they are at a friend's house or on a public computer and forget to log out?
No. Never assume that your users will do things that you want them to do if you don't explicitly tell them (i.e. force them) to do so.
The "Remember me" checkbox is an excellent example of how ignorant users can be - even when the checkbox is there, the fact that users are kept logged on on public computers is a problem. It won't help if you start assuming...
Now that most browsers can remember passwords anyway, there's little point in setting a remember-me cookie as well, so you can get rid of that option. Then users can no longer complain if they clear their cookies and then wonder why your site doesn't know them from Adam...
However, do not assume the user will log out. They don't, and with tabbed browsing, your session cookie could stick around for a very long time - maybe several days if they keep using hibernate/sleep mode and only restart their browser when it crashes. Therefore you need to set a reasonable inactivity timeout and require them to log back in once it expires.
No. Don't forget the old phrase:
When you make an assumption, you make
an "ass" out of "u" and "mption".
Thought I would update this discussion for anyone who finds this from Europe, where the EU Cookie Law will come into effect May 25th 2012. The law will require all European websites to obtain informed consent from visitors before they can store or retrieve any information on a computer or any other web connected device.
As the "Remember Me" almost certainly necessitates using a cookie, you absolutely CAN NOT assume that users will want to be remembered, and in fact doing this will become illegal from May 25th unless you somehow otherwise gain consent for cookie use from your users.
There are various strategies for how sites will comply with the new law, a popular choice currently is to have a site wide cookie Opt-In prominently displayed on your site, which disables all cookie operations until the user activates them. That's a discussion for another thread though, the message here is that you can't assume if you your web site operates out of Europe.
If your website is not security-relevant (that is: You're not a bank, medical or insurance company), then I would reverse the checkbox. Make "Remember me" the default and have a checkbox "This us a public computer, do not store my password". That would be my wish as a user, as I hate "Remember me" checkboxes, also because they don't work most of the time.
Is it safe to assume that a user wants
to be remembered when they login?
I would say that yes, in 80% of the cases (total guess on my part), the person connecting to your website is not through a public computer, and they would rather not have to retype their information every
time they come to the website.
And if they are using a public
computer, is it safe to assume that
they are smart enough to logout before
leaving?
No, that's not safe to assume. And smartness isn't even the proper term - it's more a question of remembering or even being aware I would say.
So I say go the safe route. Provide a "Remember me" checkbox, and leave it unchecked by default. People are used to this scenario. It's not that big a deal and it's much less likely to cause you trouble than checking it by default, or even worse not providing it and doing it silently.
I think it should be there on non-vital sites, but never checked by default and never automatically assumed to be wanted. If people don't want to remember their logins or retype them, let the browser's password manager or whatever deal with it, not the site. Users generally don't logout - to many, just closing the browser counts.