I'm using randomTrips.py in SUMO to generate random trips on win8. I have a map.net.xml file and try to create a trips.xml file through randomTrips.py. However, the problem occurs and I don't know how to deal with it. The code is as follows:
C:\Program Files (x86)\Eclipse\sumo\tools>randomTrips.py -n map.net.xml -l 200 -e -o map.trips.xml
I don't get the .trips.xml file I want. And the outcome is as follows, it seems that I have missed some properties of the function in my code, but I don't know how to correct it. If anyone knows how to solve the problem, pls give me some valuable suggestions. Thanks.
The outcome is :
Usage: randomTrips.py [options]
Options:
-h, --help show this help message and exit
-n NETFILE, --net-file=NETFILE
define the net file (mandatory)
-a ADDITIONAL, --additional-files=ADDITIONAL
define additional files to be loaded by the rout
-o TRIPFILE, --output-trip-file=TRIPFILE
define the output trip filename
-r ROUTEFILE, --route-file=ROUTEFILE
generates route file with duarouter
--weights-prefix=WEIGHTSPREFIX
loads probabilities for being source, destinatio
via-edge from the files named .src.xml,
.sink.xml and .via.xml
--weights-output-prefix=WEIGHTS_OUTPREFIX
generates weights files for visualisation
--pedestrians create a person file with pedestrian trips inste
vehicle trips
--persontrips create a person file with person trips instead o
vehicle trips
--persontrip.transfer.car-walk=CARWALKMODE
Where are mode changes from car to walking allow
(possible values: 'ptStops', 'allJunctions' and
combinations)
--persontrip.walkfactor=WALKFACTOR
Use FLOAT as a factor on pedestrian maximum spee
during intermodal routing
--prefix=TRIPPREFIX prefix for the trip ids
-t TRIPATTRS, --trip-attributes=TRIPATTRS
additional trip attributes. When generating
pedestrians, attributes for and
supported.
--fringe-start-attributes=FRINGEATTRS
additional trip attributes when starting on a fr
-b BEGIN, --begin=BEGIN
begin time
-e END, --end=END end time (default 3600)
-p PERIOD, --period=PERIOD
Generate vehicles with equidistant departure tim
period=FLOAT (default 1.0). If option --binomial
used, the expected arrival rate is set to 1/peri
-s SEED, --seed=SEED random seed
-l, --length weight edge probability by length
-L, --lanes weight edge probability by number of lanes
--speed-exponent=SPEED_EXPONENT
weight edge probability by speed^ (defaul
--fringe-factor=FRINGE_FACTOR
multiply weight of fringe edges by (defa
--fringe-threshold=FRINGE_THRESHOLD
only consider edges with speed above as
edges (default 0)
--allow-fringe Allow departing on edges that leave the network
arriving on edges that enter the network (via
turnarounds or as 1-edge trips
--allow-fringe.min-length=ALLOW_FRINGE_MIN_LENGTH
Allow departing on edges that leave the network
arriving on edges that enter the network, if the
at least the given length
--min-distance=MIN_DISTANCE
require start and end edges for each trip to be
least m apart
--max-distance=MAX_DISTANCE
require start and end edges for each trip to be
most m apart (default 0 which disables a
checks)
-i INTERMEDIATE, --intermediate=INTERMEDIATE
generates the given number of intermediate way p
--flows=FLOWS generates INT flows that together output vehicle
the specified period
--maxtries=MAXTRIES number of attemps for finding a trip which meets
distance constraints
--binomial=N If this is set, the number of departures per sec
will be drawn from a binomial distribution with
and p=PERIOD/N where PERIOD is the argument give
option --period. Tnumber of attemps for finding
which meets the distance constraints
-c VCLASS, --vclass=VCLASS, --edge-permission=VCLASS
only from and to edges which permit the given ve
class
--vehicle-class=VEHICLE_CLASS
The vehicle class assigned to the generated trip
(adds a standard vType definition to the output
--validate Whether to produce trip output that is already c
for connectivity
-v, --verbose tell me what you are doing
Probably the file name association with .py files is broken, see Python Command Line Arguments (Windows). Try to run the script with python explicitly:
python randomTrips.py -n map.net.xml -l 200 -e -o map.trips.xml
I just tried last week. You can search randomTrips.py under SUMO's folder. Then you find the location of randomTrips.py, then you open the cmd and call python to execute it. You also need to specify the net.xml.
Related
I'm able to get the stats (additions, deletions, total) for each commit, however how can I get the overall #?
For example, if one MR has 30 commits, I need the net # of lines of code added\deleted which you can see in the top corner.
This # IS NOT the sum of all #'s per commit.
So, I would need an API that returns the net # of lines of code added\removed at MR level (no matter how many commits are).
For example, if I have 2 commits: 1st one adds 10 lines, and the 2nd one removes the exact same 10 lines, then the net # is 0.
Here is the scenario:
I have an MR with 30 commits.
GitLab API provides support to get the stats (lines of code added\deleted) per Commit (individually).
If I go in GitLab UI, go to the MR \ Changes, I see the # of lines added\deleted that is not the SUM of all the Commits stats that I'm getting thru API.
That's my issue.
A simpler example: let's say I have 2 commits, one adds 10 lines of code, while the 2nd commit removes the exact same 10 lines of code. Using the API, I'm getting the sum, which is 20 LOCs added. However, if I go in the GitLab UI \ Changes, it's showing me 0 (zero), which is correct; that's the net # of chgs overall. This is the inconsistency I noticed.
To do this for an MR, you would use the MR changes API and count the occurrences of lines starting with + and - in the changes[].diff fields to get the additions and deletions respectively.
Using bash with gitlab-org/gitlab-runner!3195 as an example:
GITLAB_HOST="https://gitlab.com"
PROJECT_ID="250833"
MR_ID="3195"
URL="${GITLAB_HOST}/api/v4/projects/${PROJECT_ID}/merge_requests/${MR_ID}/changes"
DIFF=$(curl ${URL} | jq -r ".changes[].diff")
ADDITIONS=$(grep -E "^\+" <<< "$DIFF")
DELETIONS=$(grep -E "^\-" <<< "$DIFF")
NUM_ADDITIONS=$(wc -l <<< "$ADDITIONS")
NUM_DELETIONS=$(wc -l <<< "$DELETIONS")
echo "${MR_ID} has ${NUM_ADDITIONS} additions and ${NUM_DELETIONS} deletions"
The output is
3195 has 9 additions and 2 deletions
This matches the UI, which also shows 9 additions and 2 deletions
This, as you can see is a representative example of your described scenario since the combined total of the individual commits in this MR are 13 additions and 6 deletions.
I'm trying to base my cluster memory allocation for a given rule on the file size of an input file. Is this possible in snakemake and if so how?
So far I have tried specifying it in the resource: section like so:
rule compute2:
input: "input1.txt"
output: "input2.txt"
resources:
mem_mb=lambda wildcards, input, attempt: int(os.path.getsize(str(input))/(1024*1024))
shell: "touch input2.txt"
But it seems snakemake attempts to calculate this upfront before the file gets created as I'm getting this error:
InputFunctionException in line 35 of test_snakemake/Snakefile:
FileNotFoundError: [Errno 2] No such file or directory: 'input1.txt'
Im running my snakemake with the following command:
snakemake --verbose -j 10 --cluster-config cluster.json --cluster "sbatch -n {cluster.n} -t {cluster.time} --mem {resources.mem_mb}"
If you want to do this dynamically per rule as per the question, you can use something along these lines:
resources: mem_mb=lambda wildcards, input, attempt: (input.size//1000000) * attempt * 10
Where input.size//1000000 is used convert the cumulative size of input files in bytes to mb, and the tailing 10 could be any arbitrary number based on the specifics of your shell/script requirements.
This is possible using the --default-resources option. As explained in Snakemake's --help information:
In addition to plain integers, python expressions over input size are
allowed (e.g. '2*input.size_mb'). When specifying this without any
arguments (--default-resources), it defines
'mem_mb=max(2*input.size_mb, 1000)''disk_mb=max(2*input.size_mb, 1000)',
i.e., default disk and mem usage is twice the input file size
but at least 1GB.
Hashcat doesn't support the target application I'm trying to crack, but I'm wondering whether the mask function can be 'fed' the list of passwords and parsed through the rockyou rule to generate an effective wordlist for me?
If so, how can this be done as the documentation leaves lots to be desired.. !
Many thanks
I used HashCatRulesEngine:
https://github.com/llamasoft/HashcatRulesEngine
You can chain all the HashCat rules together, it then union selects them, weeds out any duplicates and takes as input your sample password file.
It then generates all possible permutations.
For instance:
echo "password">foo
./hcre /Users/chris/Downloads/hashcat-4.0.0/rules/Incisive-leetspeak.rule /Users/chris/Downloads/hashcat-4.0.0/rules/InsidePro-HashManager.rule /Users/chris/Downloads/hashcat-4.0.0/rules/InsidePro-PasswordsPro.rule /Users/chris/Downloads/hashcat-4.0.0/rules/T0XlC-insert_00-99_1950-2050_toprules_0_F.rule /Users/chris/Downloads/hashcat-4.0.0/rules/T0XlC-insert_space_and_special_0_F.rule /Users/chris/Downloads/hashcat-4.0.0/rules/T0XlC-insert_top_100_passwords_1_G.rule /Users/chris/Downloads/hashcat-4.0.0/rules/T0XlC.rule /Users/chris/Downloads/hashcat-4.0.0/rules/T0XlCv1.rule /Users/chris/Downloads/hashcat-4.0.0/rules/best64.rule /Users/chris/Downloads/hashcat-4.0.0/rules/combinator.rule /Users/chris/Downloads/hashcat-4.0.0/rules/d3ad0ne.rule /Users/chris/Downloads/hashcat-4.0.0/rules/dive.rule /Users/chris/Downloads/hashcat-4.0.0/rules/generated.rule /Users/chris/Downloads/hashcat-4.0.0/rules/generated2.rule /Users/chris/Downloads/hashcat-4.0.0/rules/hybrid /Users/chris/Downloads/hashcat-4.0.0/rules/leetspeak.rule /Users/chris/Downloads/hashcat-4.0.0/rules/oscommerce.rule /Users/chris/Downloads/hashcat-4.0.0/rules/rockyou-30000.rule /Users/chris/Downloads/hashcat-4.0.0/rules/specific.rule /Users/chris/Downloads/hashcat-4.0.0/rules/toggles1.rule /Users/chris/Downloads/hashcat-4.0.0/rules/toggles2.rule /Users/chris/Downloads/hashcat-4.0.0/rules/toggles3.rule /Users/chris/Downloads/hashcat-4.0.0/rules/toggles4.rule /Users/chris/Downloads/hashcat-4.0.0/rules/toggles5.rule /Users/chris/Downloads/hashcat-4.0.0/rules/unix-ninja-leetspeak.rule < foo >passwordsx
1 password the word "password" was permutated a total of:
bash-3.2# wc -l passwordsx
227235 passwordsx
bash-3.2#
Times meaning that each word you feed into this generates 227235 possible combinations roughly giving you full coverage..
You can use hashcat itself as a candidate generator by adding the --stdout switch (then pipe to your file or program of choice). I haven't tried all the possibilities, but it should work with any of the supported hashcat modes.
Here's an example using a ruleset: https://hashcat.net/wiki/doku.php?id=rule_based_attack#debugging_rules
I have a bunch of instances running in GCE. I want to programmatically get a list of the internal IP addresses of them without logging into the instances (locally).
I know I can run:
gcloud compute instances list
But are there any flags I can pass to just get the information I want?
e.g.
gcloud compute instances list --internal-ips
or similar? Or am I going to have to dust off my sed/awk brain and parse the output?
I also know that I can get the output in JSON using --format=json, but I'm trying to do this in a bash script.
The simplest way to programmatically get a list of internal IPs (or external IPs) without a dependency on any tools other than gcloud is:
$ gcloud --format="value(networkInterfaces[0].networkIP)" compute instances list
$ gcloud --format="value(networkInterfaces[0].accessConfigs[0].natIP)" compute instances list
This uses --format=value which also requires a projection which is a list of resource keys that select resource data values. For any command you can use --format=flattened to get the list of resource key/value pairs:
$ gcloud --format=flattened compute instances list
A few things here.
First gcloud's default output format for listing is not guaranteed to be stable, and new columns may be added in the future. Don't script against this!
The three output modes are three output modes that are accessible with the format flag, --format=json, --format=yaml, and format=text, are based on key=value pairs and can scripted against even if new fields are introduced in the future.
Two good ways to do what you want are to use JSON and the jq tool,
gcloud compute instances list --format=json \
| jq '.[].networkInterfaces[].networkIP'
or text format and grep + line-oriented using tools,
gcloud compute instances list --format=text \
| grep '^networkInterfaces\[[0-9]\+\]\.networkIP:' | sed 's/^.* //g'
I hunted around and couldn't find a straight answer, probably because efficient tools weren't available when others replied to the original question. GCP constantly updates their libraries & APIs and we can use the filter and projections to extract targeted attributes.
Here I outline how to reserve an external static IP, see how it's attributes are named & organised, and then export the external IP address so that I can use it in other scripts (e.g. assign this to a VM instance or authorise this network (IP address) on a Cloud SQL instance.
Reserve a static IP in a region of your choice
gcloud compute --project=[PROJECT] addresses create [NAME] --region=[REGION]
[Informational] View the details of the regional static IP that was reserved
gcloud compute addresses describe [NAME] --region [REGION] --format=flattened
[Informational] List the attributes of the static IP in the form of key-value pairs
gcloud compute addresses describe [NAME] --region [REGION] --format='value(address)'
Extract the desired value (e.g. external IP address) as a parameter
export STATIC_IP=$(gcloud compute addresses describe [NAME] --region [REGION] --format='value(address)’)
Use the exported parameter in other scripts
echo $STATIC_IP
The best possible way would be to have readymade gcloud command use the same as and when needed.
This can be achieved using table() format option with gcloud as per below:
gcloud compute instances list --format='table(id,name,status,zone,networkInterfaces[0].networkIP :label=Internal_IP,networkInterfaces[0].accessConfigs[0].natIP :label=External_IP)'
What does it do for you?
Get you data in clean format
Give you option to add or remove columns
Need additional columns? How to find column name even before you run the above command?
Execute the following, which will give you data in raw JSON format consisting value and its name, copy those names and add them into your table() list. :-)
gcloud compute instances list --format=json
Plus Point: This is pretty much same syntax you can tweak with any GCP resources data to fetch including with gcloud, kubectl etc.
As far as I know you can't filter on specific fields in the gcloud tool.
Something like this will work for a Bash script, but it still feels a bit brittle:
gcloud compute instances list --format=yaml | grep " networkIP:" | cut -c 14-100
I agree with #Christiaan. Currently there is no automated way to get the internal IPs using the gcloud command.
You can use the following command to print the internal IPs (4th column):
gcloud compute instances list | tail -n+2 | awk '{print $4}'
or the following one if you want to have the pair <instance_name> <internal_ip> (1st and 4th column)
gcloud compute instances list | tail -n+2 | awk '{print $1, $4}'
I hope it helps.
I have some fairly simple Hadoop streaming jobs that look like this:
yarn jar /usr/lib/hadoop-mapreduce/hadoop-streaming-2.2.0.2.0.6.0-101.jar \
-files hdfs:///apps/local/count.pl \
-input /foo/data/bz2 \
-output /user/me/myoutput \
-mapper "cut -f4,8 -d," \
-reducer count.pl \
-combiner count.pl
The count.pl script is just a simple script that accumulates counts in a hash and prints them out at the end - the details are probably not relevant but I can post it if necessary.
The input is a directory containing 5 files encoded with bz2 compression, roughly the same size as each other, for a total of about 5GB (compressed).
When I look at the running job, it has 45 mappers, but they're all running on one node. The particular node changes from run to run, but always only one node. Therefore I'm achieving poor data locality as data is transferred over the network to this node, and probably achieving poor CPU usage too.
The entire cluster has 9 nodes, all the same basic configuration. The blocks of the data for all 5 files are spread out among the 9 nodes, as reported by the HDFS Name Node web UI.
I'm happy to share any requested info from my configuration, but this is a corporate cluster and I don't want to upload any full config files.
It looks like this previous thread [ why map task always running on a single node ] is relevant but not conclusive.
EDIT: at #jtravaglini's suggestion I tried the following variation and saw the same problem - all 45 map jobs running on a single node:
yarn jar \
/usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples-2.2.0.2.0.6.0-101.jar \
wordcount /foo/data/bz2 /user/me/myoutput
At the end of the output of that task in my shell, I see:
Launched map tasks=45
Launched reduce tasks=1
Data-local map tasks=18
Rack-local map tasks=27
which is the number of data-local tasks you'd expect to see on one node just by chance alone.