I want to use the ZAP API to perform authenticated scans against a number of different web applications. These web applications each have different mechanisms to login and I do not want to perform the tedious process of logging in via a number of different forms which each need to be manually configured.
The easier solution is to use the HTTP session cookies for each application to perform these authenticated scans, however I cannot see a mechanism to perform this without creating a context with an associated user.
I attempted to add a new session via the http sessions, despite them not being for this use case:
def add_session(self, session_name, session_tokens):
self.zap.httpsessions.create_empty_session(self.target_url, session_name, apikey=self.api_key)
for token_name, token_value in session_tokens:
self.zap.httpsessions.set_session_token_value(self.target_url, session_name, token_name, token_value, apikey=self.api_key)
self.zap.httpsessions.add_session_token(self.target_url, session_name, apikey=self.api_key)
self.zap.httpsessions.set_active_session(self.target_url, session_name, apikey=self.api_key)
However, when performing a scan any manually added cookies are not added to the subsequent requests to the server.
For example, when performing a spider the session information is ignored:
def spider(self):
scanid = self.zap.spider.scan(self.target_url, apikey=self.api_key)
while (int(self.zap.spider.status(scanid)) < 100):
print 'Spider progress %s%% ' % self.zap.spider.status(scanid)
time.sleep(1.0)
return self.zap.spider.full_results(scanid)
Is it possible to perform scans by adding cookies to requests via the ZAP API?
Or is the only option to manually add the form data and context for each website to which I want to login and scan?
Yes it is possible - we do this at Mozilla. The snippet of code I use is:
zap.httpsessions.add_session_token(target + ":443", "sessionid")
zap.httpsessions.create_empty_session(target + ":443", "testsession")
zap.httpsessions.set_session_token_value(target + ":443", "testsession", "sessionid", session_cookie)
zap.httpsessions.set_active_session(target + ":443", "testsession")
That works with both spidering and active scanning, no context required. I use the Auth Stats (https://github.com/zaproxy/zap-extensions/wiki/HelpAddonsAuthstatsAuthStats) add-on to check that the authentication is working as expected.
Cheers, Simon
Related
In HTTP Request, I gave login credentials then login got success, access_token generated.
Then by using JSON Extractor, I extracted the access_token and name as "auth_token".
Then in Beanshell Assertion, I added script "${__setProperty(auth_token, ${auth_token})};"
Then in HTTP Request Defaults, I added Parameter "Authorization" "${__property(auth_token)}" (in Test Plan level)
I cannot fetch that token in Thread_group-2, which is generated in Thread_group-1.
suggestions, please??
You need to ensure that the Thread Group which extracts the token is executed before the Thread Group which uses the token, either tick "Run Thread Groups consecutively" on Test Plan level:
or switch to setUp Thread Group which is being executed before all "main" Thread Groups
In general you should not be passing values between Thread Groups as thread group represents a logical group of business users and authentication/authorization is an integral part of the business flow so my expectation is that you need to have authentication and other actions in the single Thread Group. If you want to perform authentication only once - put it under Once Only Controller
Using Beanshell is kind of a performance anti-pattern, consider moving to JSR223 Test Elements instead.
I also have doubts regarding correctness of your token usage, I think you should rather use HTTP Header Manager instead and configure it to pass Authorization header with the value of Bearer followed by whitespace followed by your token
I'm accessing a external rest api which is secured using OpenID connect. I must run a job which calls the api several times before it completes. This job may be executed in parallel using several concurrent threads. Thus I use a service for api access which is instantiated for each job:
private AccessAndRefresTokens tokens; // I initially get that from somewhere else
public Mono<Result> callTheApi(){
return createWebClientWith(tokens.accessToken).executeRequest();
}
public Mono<Result> callOtherApiFunction(){
return createWebClientWith(tokens.accessToken).executeRequest();
}
public Mono<Result> callYetAnotherApiFunction(){
return createWebClientWith(tokens.accessToken).executeRequest();
}
As my job is executed it might happen that the access token expires between two api calls. To prevent this from happening, I like to check the validity of the access token before every request and refresh it if necessary.
My first idea was to do the validity check inside a flatMap operator:
Mono.just(tokens).flatMap(tokens -> {
if(accessTokenExpired){
return refreshAccessToken().doOnNext(refreshedTokens -> tokens = refreshedTokens);
}else{
return Mono.just(tokens.accessToken);
}
}).flatMap(accessToken -> createWebClientWith(accessToken));
However it seems to me that this would trigger the refresh several times if multiple threads access the service and the refresh has not yet completed. As a consequence I would end up refreshing the access token several times in a very short time which might fail due to rate limits.
I am new to the whole reactive thing and I suppose using a synchronized block is not a desired option. So I tried to figure out something using reactor's Processor but I could not find a satisfying solution.
So is there a way to ensure the access token is refershed only once? And how do I achieve this without using blocking code?
When asking this question I am looking for guidance with implementation of my own AuthenticationProvider. By that i mean the following:
Till now i have learned that Spring Security ask the AuthenticationProvider object if the user is authenticated or not. Currently i am using the
DaoAuthenticationProvider wich process the username and the password that is beeing returned by my own custome UserDetailService. All works great!
Spring support a lot of different AuthenticationProvider's for example there is one for LDAP, Jdbc, DAO (as mentioned above) and i was even able to find one for Kerberos. However there is no authentication provider for SRP, thus the need to write one.
My question is here the following:
When we use the DaoAuthenticationProvider i.e. user/password authentication, we have a html form where the username and password are entered and then a button is responsible for submiting those 2 parameters. Bought barameteres are transfered over some transport channel to the server i.e. with one click we are able to send all of the data within the same http reqest i.e. all that is needed for the authentication. That can be seen in the UsernamePasswordAuthenticationFilter. Here the method "attemptAuthentication" takes "HttpServletRequest request" which includes username and password. Till now all good.
public Authentication attemptAuthentication(HttpServletRequest request, HttpServletResponse response) throws AuthenticationException {
...
}
Well, in the simple in SRP we have also a form that has a username and password, except the password !!!MUST!!! not be transfered over the network. In order to achieve that constrain a "discussion" between the client and the server has to take place i.e. the following parameters must be exchanged.
1) username (I),
2) a value called "A"
3) a value called "B"
4) a value called "M1"
5) a value called "M2"
Well, let us assume that there is a filter called "SrpAuthenticationProcessingFilter" how should the filter look like if the new authentication protocol was more like a conversation by that i mean from the RFC 5054.
Client Server
Client Hello (I) -------->
Server Hello
Certificate*
Server Key Exchange (N, g, s, B)
<-------- Server Hello Done
Client Key Exchange (A) -------->
[Change cipher spec]
Finished -------->
[Change cipher spec]
<-------- Finished
Application Data <-------> Application Data
Here we have a client that needs to
a) send first his username (I)
b) then the server needs to respond with a value B. (N,g,s are not really necessary in this case)
c) the client sends it's "value A"
d) the the client uses that value B from the server and based on that value calculates a key.
e) the server calculates the key as well based on the value A.
f) the client sends value M1 to the server.
g) the server gets the M1 value and based on a formula which has the M1 value as argument, he is able to verify if the keys
match, if the calculated keys of the bought sides match then the
user is authenticated, and the product i.e. the shared key could
be used further for other processing.
In a contrast to username and password authentication those are 7 steps not one. And 3 of those needs to happen before the SrpAuthenticationProcessingFilter. Now i am aware that there is a possibility to send the username together with the "value A" thus shortening the number of steps, but i would like to strictly follow the RFC. Never take the easy way right?
The question really is where do i place that code which is responcible for the ping pong (conversation) between the client and the server i.e. the first 3 steps a,b and c mentioned above. Should it place them in a SrpEntryPoint object, or somewhere else. if else then were in the context of SpringSecurity?
One way i can thing of solving this is using websockets, but i would also like to make that authentication independent from any layer 5-7 protocol such as websockets, http, spdy etc. . That means that the first implementation should be be trough simple http request/responce and then with any other protocol.
So the structure that could be the right one in order to implement SRP currently is:
SRPAuthenticationEntryPoint implements org.springframework.security.web.AuthenticationEntryPoint - this basically says what should be done if a request for a protected resource gets in, but the user is not authenticated yet. Here we can place a redirect in the case the resource has not been authenticated. Maybe this is also the place which is responsible for the steps a,b,c not sure if that is the right place. Request guidance and information!!
SrpAuthenticationProcessingFilter extends GenericFilterBean. SrpAuthenticationProcessingFilter exists to make partial validation for example to check if the srp paramters received a correct and corrspoding to the srp parameters that the server sets. Here it is important to mention that SrpAuthenticationProcessingFilter is not doing any username validation i.e. that needs to happen in one step before the SrpAuthenticationProcessingFilter is beeing called, maybe that is the SrpEntryPoint or some other step that i do not know how to call it yet. SrpAuthenticationProcessingFilter has a method "doFilter" in which the second structure is beeing created i.e. SrpAuthenticationToken.
SrpAuthenticationToken extend org.springframework.security.authentication.AbstractAuthenticationToken. That token in my understanding is something similar ot the DAO object, the token maps all field needed for sucessefull authentication. When partially validated parameters are populated into the SrpAuthenticationToken that SrpAuthenticationToken is passed to the authenticat method of the org.springframework.security.authentication.AuthenticationManager interface i.e. something like that
myAuthentication = authenticationManager.authenticate(SrpAuthenticationToken);
depending on which Authentication Provider is configured in the Spring Security config then the SrpAuthentication provider is called in our case i.e.:
#Autowired
public void registerAuthentication(AuthenticationManagerBuilder auth) throws Exception {
auth
.authenticationProvider(sRPAuthenticationProvider);
}
SRPAuthenticationProvider - implements org.springframework.security.authentication.AuthenticationProvider. Over here the steps d,e,f and g are being validated and compared. If something wrong happens then throw new BadCredentialsException("Invalid username/password/Verifiere") exception.
SrpSessionValidator - this one is responsible only for validating particular parameters of the Srp session and would be called from SrpAuthenticationProcessingFilter as well as in one step before the SrpAuthenticationProcessingFilter in order to validate if the username exists in the database at all.
I only have a general idea how to implement that Srp authentication, and thus i would like some comments if that makes sense at all and if SRPAuthenticationEntryPoint is the right place for the steps a,b and c. It does not feel like the right place to me.
Any feedback is greatly appreciated.
regards,
Tito
Addition1 (21 November 2014) -> As an aswer to the question "where to place that code which is responsible for the ping pong (conversation) between the client and the server i.e. the first 3 steps a,b and c " the answer to that would be most likely a standard (i will call it negotiation) filter which will take that job.
Now I would like to rephrase the question i.e. before the authentication is complete and before the M1 and M2 messages are received i.e. steps 1 2 and 3. Where do i place that object? I.e. it should be a place where the object should live for example for 60 seconds and then be automatically deleted in case no M1 and M2 messages has been received. I mean some object before the "SecurityContextHolder" object. I just do not know what is the name of that object/context related to spring security, also i do not know if such construct exists at all?
My approach would be to use AJAX to run the first parts of the protocol up to the creation of A and M1 at the client then post those to the server as the login credentials and check those using Spring Security.
To see how this works there is a junit-js test which runs mutual authentication between a javascript client object and a java server object using Thinbus over at TestSRP6JavascriptClientSessionSHA256.js (note the maven build runs this unit test using the JDK javascript runtime):
// note the following exchange is written in javascript calling client js and invoking server java which is run by JUnit-JS using the JDK javascript runtime so it shows both sides of the full authentication in one unit test method
// normal login flow step1a client: browser starts with username and password given by user at the browser
client.step1(username,password);
// normal login flow step1b server: server starts with username from browser plus salt and verifier saved to database on user registration.
var B = server.step1(username, salt, v);
// normal login flow step2a client: server sends users salt from user registration and the server ephemeral number
var credentials = client.step2(salt, B);
// normal login flow step2b server: client sends its client ephemeral number and proof of a shared session key derived from both ephermal numbers and the password
var M2 = server.step2(credentials.A, credentials.M1);
// normal login flow step3 client: client verifies that the server shows proof of the shared session key which demonstrates that it knows actual verifier
client.step3(M2);
Clearly the javascript client starts with only username and password. The server uses the username to resolve the salt and generates a random B. The client is given salt and B from the server and generates its random A and the M1 which is the password proof. The server step2 which takes M1 as a parameter is the server checking the user password proof and it will throw an exception if the proof is bad. The server then sends M2 which is the server proof that it has the user verifier v which is done to prevent a fake server spoofing the real server.
There is a demo of a browser doing SRP to a Java server over AJAX using Thinbus over at thinbus-srp-js-demo. You could re-use the JAX RS with AJAX approach (e.g. Spring MVC AJAX) to perform the first steps up to the creation of A+M1 at the client then post those to Spring Security using the login form post and have Spring Security verify A+M1 by running step2 on the server object as shown in the junit-js test. Then your AuthenticationManager could resolve the server object created by AJAX from a concurrent map keyed by username.
One minor note is that I would consider checking the server proof M2 as optional if you are using HTTPS to a server. If you are not using HTTPS then server spoofing means they can give the user a page which sends the password and ignores a bad M2; so the M2 proof doesn't provide security in the context of webpages. A mobile app which packages the html/js into a native app using something like phone gap would benefit from an M2 check; it can be added into the page after the user is logged in to be checked by the trusted code.
There is a spring security SRP implimentation at https://bitbucket.org/simon_massey/thinbus-srp-spring-demo
The readme page says:
A salient feature of the integration is that there are no changes to
the standard spring security authentication path other than to
configure the custom SrpAuthenticationProvider.
It also says:
Spring Security does not expose the user HTTPSession to the
AuthenticationProvider. So demo code uses a Guava cache with a timeout
to hold the SRP6JavascriptServerSession for the brief period of the
login exchange. As of Thinbus 1.2.1 the session is serialisable so an
alternative approach for a large stateless website would be to hold
the SRP6JavascriptServerSession object in the DB rather than in an
in-memory cache.
Is there a way to test if current user has been authenticated to BBG? I have my c# program which uses BBG API, and want to check if the user logged in the service before, either via API calls or the BBG Terminal. This check can then be used to distinguish whether the user's network is unavailable or simply he hasn't logged in yet.
Thanks!
There's a couple of ways to interpret your question, so I'll answer both... (I'm speaking from the perspective of using the Java API, but it should be pretty similar on C#.)
1. Can I tell whether the user connect to Bloomberg (i.e. are there network issues / are they are logged in)?
Yes - you can create a new Session, try to start it using .start(). If it fails or returns false, you cannot connect. If it starts, you can call .openService("//blp/apiauth"). Again, if it fails or returns false, you cannot connect.
If you cannot connect, you may or may not be able to determine why you cannot... Nevertheless, I would suggest registering a callback to the BLP API logging framework. In our code, we we-direct these to the logging framework we use throughout our code.
2. The user has created a Session (pre-cursor to a Service) - can I tell if the Session has been started?
Unfortunately - no. There is nothing in the API to allow you to determine the state of the Session. (I suppose you could try starting it, and if it starts it wasn't started, and if it fails, it was started - but that strikes me as an unhelpful or risk appraoch.)
I'm thinking about the best way to create a cache layer in front or as first layer for GET requests to my RESTful API (written in Ruby).
Not every request can be cached, because even for some GET requests the API has to validate the requesting user / application. That means I need to configure which request is cacheable and how long each cached answer is valid. For a few cases I need a very short expiration time of e.g. 15s and below. And I should be able to let cache entries expire by the API application even if the expiration date is not reached yet.
I already thought about many possible solutions, my two best ideas:
first layer of the API (even before the routing), cache logic by myself (to have all configuration options in my hand), answers and expiration date stored to Memcached
a webserver proxy (high configurable), perhaps something like Squid but I never used a proxy for a case like this before and I'm absolutely not sure about it
I also thought about a cache solution like Varnish, I used Varnish for "usual" web applications and it's impressive but the configuration is kind of special. But I would use it if it's the fastest solution.
An other thought was to cache to the Solr Index, which I'm already using in the data layer to not query the database for most requests.
If someone has a hint or good sources to read about this topic, let me know.
Firstly, build your RESTful API to be RESTful. That means authenticated users can also get cached content as to keep all state in the URL it needs to contain the auth details. Of course the hit rate will be lower here, but it is cacheable.
With a good deal of logged in users it will be very beneficial to have some sort of model cache behind a full page cache as many models are still shared even if some aren't (in a good OOP structure).
Then for a full page cache you are best of to keep all the requests off the web server and especially away from the dynamic processing in the next step (in your case Ruby). The fastest way to cache full pages from a normal web server is always a caching proxy in front of the web servers.
Varnish is in my opinion as good and easy as it gets, but some prefer Squid indeed.
memcached is a great option, and I see you mentioned this already as a possible option. Also Redis seems to be praised a lot as another option at this level.
On an application level, in terms of a more granular approach to cache on a file by file and/or module basis, local storage is always an option for common objects a user may request over and over again, even as simple as just dropping response objects into session so that can be reused vs making another http rest call and coding appropriately.
Now people go back and forth debating about varnish vs squid, and both seem to have their pros and cons, so I can't comment on which one is better but many people say Varnish with a tuned apache server is great for dynamic websites.
Since REST is an HTTP thing, it could be that the best way of caching requests is to use HTTP caching.
Look into using ETags on your responses, checking the ETag in requests to reply with '304 Not Modified' and having Rack::Cache to serve cached data if the ETags are the same. This works great for cache-control 'public' content.
Rack::Cache is best configured to use memcache for its storage needs.
I wrote a blog post last week about the interesting way that Rack::Cache uses ETags to detect and return cached content to new clients: http://blog.craz8.com/articles/2012/12/19/rack-cache-and-etags-for-even-faster-rails
Even if you're not using Rails, the Rack middleware tools are quite good for this stuff.
Redis Cache is best option.
check here.
It is open source. Advanced key-value cache and store.
I’ve used redis successfully this way in my REST view:
from django.conf import settings
import hashlib
import json
from redis import StrictRedis
from django.utils.encoding import force_bytes
def get_redis():
#get redis connection from RQ config in settings
rc = settings.RQ_QUEUES['default']
cache = StrictRedis(host=rc['HOST'], port=rc['PORT'], db=rc['DB'])
return cache
class EventList(ListAPIView):
queryset = Event.objects.all()
serializer_class = EventSerializer
renderer_classes = (JSONRenderer, )
def get(self, request, format=None):
if IsAdminUser not in self.permission_classes: # dont cache requests from admins
# make a key that represents the request results you want to cache
# your requirements may vary
key = get_key_from_request()
# I find it useful to hash the key, when query parms are added
# I also preface event cache key with a string, so I can clear the cache
# when events are changed
key = "todaysevents" + hashlib.md5(force_bytes(key)).hexdigest()
# I dont want any cache issues (such as not being able to connect to redis)
# to affect my end users, so I protect this section
try:
cache = get_redis()
data = cache.get(key)
if not data:
# not cached, so perform standard REST functions for this view
queryset = self.filter_queryset(self.get_queryset())
serializer = self.get_serializer(queryset, many=True)
data = serializer.data
# cache the data as a string
cache.set(key, json.dumps(data))
# manage the expiration of the cache
expire = 60 * 60 * 2
cache.expire(key, expire)
else:
# this is the place where you save all the time
# just return the cached data
data = json.loads(data)
return Response(data)
except Exception as e:
logger.exception("Error accessing event cache\n %s" % (e))
# for Admins or exceptions, BAU
return super(EventList, self).get(request, format)
in my Event model updates, I clear any event caches.
This hardly ever is performed (only Admins create events, and not that often),
so I always clear all event caches
class Event(models.Model):
...
def clear_cache(self):
try:
cache = get_redis()
eventkey = "todaysevents"
for key in cache.scan_iter("%s*" % eventkey):
cache.delete(key)
except Exception as e:
pass
def save(self, *args, **kwargs):
self.clear_cache()
return super(Event, self).save(*args, **kwargs)