drwxr-xr-x - mgupta supergroup 0 2018-03-26 22:16 /kylin/retailer/qi_basket_brand_bucket_fact/_impala_insert_staging
drwxr-xr-x - mgupta supergroup 0 2018-03-26 22:18 /kylin/retailer/qi_basket_brand_bucket_fact/product_hierarchy_type=CI
[mgupta#sjc-dev-binn01 ~]$ hadoop fs -ls /kylin/retailer/qi_basket_brand_bucket_fact/product_hierarchy_type=CI
Found 27 items
drwxr-xr-x - mgupta supergroup 0 2018-03-26 22:16 /kylin/retailer/qi_basket_brand_bucket_fact/product_hierarchy_type=CI/month_id=201601
drwxr-xr-x - mgupta supergroup 0 2018-03-26 22:16 /kylin/retailer/qi_basket_brand_bucket_fact/product_hierarchy_type=CI/month_id=201602
drwxr-xr-x - mgupta supergroup 0 2018-03-26 22:16 /kylin/retailer/qi_basket_brand_bucket_fact/product_hierarchy_type=CI/month_id=201603
drwxr-xr-x - mgupta supergroup 0 2018-03-26 22:16 /kylin/retailer/qi_basket_brand_bucket_fact/product_hierarchy_type=CI/month_id=201604
drwxr-xr-x - mgupta supergroup 0 2018-03-26 22:16 /kylin/retailer/qi_basket_brand_bucket_fact/product_hierarchy_type=CI/month_id=201605
drwxr-xr-x - mgupta supergroup 0 2018-03-26 22:16 /kylin/retailer/qi_basket_brand_bucket_fact/product_hierarchy_type=CI/month_id=201606
drwxr-xr-x - mgupta supergroup 0 2018-03-26 22:16 /kylin/retailer/qi_basket_brand_bucket_fact/product_hierarchy_type=CI/month_id=201607
drwxr-xr-x - mgupta supergroup 0 2018-03-26 22:16 /kylin/retailer/qi_basket_brand_bucket_fact/product_hierarchy_type=CI/month_id=201608
drwxr-xr-x - mgupta supergroup 0 2018-03-26 22:17 /kylin/retailer/qi_basket_brand_bucket_fact/product_hierarchy_type=CI/month_id=201609
drwxr-xr-x - mgupta supergroup 0 2018-03-26 22:17 /kylin/retailer/qi_basket_brand_bucket_fact/product_hierarchy_type=CI/month_id=201610
drwxr-xr-x - mgupta supergroup 0 2018-03-26 22:17 /kylin/retailer/qi_basket_brand_bucket_fact/product_hierarchy_type=CI/month_id=201611
drwxr-xr-x - mgupta supergroup 0 2018-03-26 22:17 /kylin/retailer/qi_basket_brand_bucket_fact/product_hierarchy_type=CI/month_id=201612
drwxr-xr-x - mgupta supergroup 0 2018-03-26 22:17 /kylin/retailer/qi_basket_brand_bucket_fact/product_hierarchy_type=CI/month_id=201701
drwxr-xr-x - mgupta supergroup 0 2018-03-26 22:17 /kylin/retailer/qi_basket_brand_bucket_fact/product_hierarchy_type=CI/month_id=201702
drwxr-xr-x - mgupta supergroup 0 2018-03-26 22:17 /kylin/retailer/qi_basket_brand_bucket_fact/product_hierarchy_type=CI/month_id=201703
drwxr-xr-x - mgupta supergroup 0 2018-03-26 22:17 /kylin/retailer/qi_basket_brand_bucket_fact/product_hierarchy_type=CI/month_id=201704
drwxr-xr-x - mgupta supergroup 0 2018-03-26 22:17 /kylin/retailer/qi_basket_brand_bucket_fact/product_hierarchy_type=CI/month_id=201705
drwxr-xr-x - mgupta supergroup 0 2018-03-26 22:17 /kylin/retailer/qi_basket_brand_bucket_fact/product_hierarchy_type=CI/month_id=201706
drwxr-xr-x - mgupta supergroup 0 2018-03-26 22:17 /kylin/retailer/qi_basket_brand_bucket_fact/product_hierarchy_type=CI/month_id=201707
drwxr-xr-x - mgupta supergroup 0 2018-03-26 22:18 /kylin/retailer/qi_basket_brand_bucket_fact/product_hierarchy_type=CI/month_id=201708
drwxr-xr-x - mgupta supergroup 0 2018-03-26 22:18 /kylin/retailer/qi_basket_brand_bucket_fact/product_hierarchy_type=CI/month_id=201709
drwxr-xr-x - mgupta supergroup 0 2018-03-26 22:18 /kylin/retailer/qi_basket_brand_bucket_fact/product_hierarchy_type=CI/month_id=201710
drwxr-xr-x - mgupta supergroup 0 2018-03-26 22:18 /kylin/retailer/qi_basket_brand_bucket_fact/product_hierarchy_type=CI/month_id=201711
drwxr-xr-x - mgupta supergroup 0 2018-03-26 22:18 /kylin/retailer/qi_basket_brand_bucket_fact/product_hierarchy_type=CI/month_id=201712
drwxr-xr-x - mgupta supergroup 0 2018-03-26 22:18 /kylin/retailer/qi_basket_brand_bucket_fact/product_hierarchy_type=CI/month_id=201801
drwxr-xr-x - mgupta supergroup 0 2018-03-26 22:18 /kylin/retailer/qi_basket_brand_bucket_fact/product_hierarchy_type=CI/month_id=201802
drwxr-xr-x - mgupta supergroup 0 2018-03-26 22:18 /kylin/retailer/qi_basket_brand_bucket_fact/product_hierarchy_type=CI/month_id=201803
[mgupta#sjc-dev-binn01 ~]$ hadoop fs -ls /kylin/retailer/qi_basket_brand_bucket_fact/product_hierarchy_type=CI/month_id=201601
Found 3 items
drwxr-xr-x - mgupta supergroup 0 2018-03-26 22:16 /kylin/retailer/qi_basket_brand_bucket_fact/product_hierarchy_type=CI/month_id=201601/company_sid=0
drwxr-xr-x - mgupta supergroup 0 2018-03-26 22:16 /kylin/retailer/qi_basket_brand_bucket_fact/product_hierarchy_type=CI/month_id=201601/company_sid=38527
drwxr-xr-x - mgupta supergroup 0 2018-03-26 22:16 /kylin/retailer/qi_basket_brand_bucket_fact/product_hierarchy_type=CI/month_id=201601/company_sid=__HIVE_DEFAULT_PARTITION__
[mgupta#sjc-dev-binn01 ~]$ hadoop fs -ls /kylin/retailer/qi_basket_brand_bucket_fact/product_hierarchy_type=CI/month_id=201601/company_sid=0
Found 1 items
-rw-r--r-- 3 mgupta supergroup 2069014 2018-03-26 22:16 /kylin/retailer/qi_basket_brand_bucket_fact/product_hierarchy_type=CI/month_id=201601/company_sid=0/f9466a0068b906cf-6ace7f8500000049_294515768_data.0.parq
[mgupta#sjc-dev-binn01 ~]$
You may try the steps given below.
Approach 1
Identify the schema (column names and types, including the partitioned column)
Create a hive partitioned table (Make sure to add partition column & delimiter information)
Load data into the partitioned table.(In this case, the loading file will not have partition column as you will hard-code it via the load command)
create table <table_name> (col1 data_type1, col2 data_type2..)
partitioned by(part_col data_type3)
row format delimited
fields terminated by '<field_delimiter_in_your_data>'
load data inpath '/hdfs/loc/file1' into table <table_name>
partition (<part_col>='201601');
load data inpath '/hdfs/loc/file2' into table <table_name>
partition (<part_col>='201602')
load data inpath '/hdfs/loc/file3' into table <table_name>
partition (<part_col>='201603')
and so on.
Approach 2
Create a staging table (temporary table) with same schema as of main table but without any partitions
Load your entire data into this table (Make sure you have the 'partition column' as one of the fields in these files)
Load data to your main table from staging table using dynamic partition insert.
create table <staging_table> (col1 data_type1, col2 data_type2..)
row format delimited
fields terminated by '<field_delimiter_in_your_data>'
create table <main_table> (col1 data_type1, col2 data_type2..)
partitioned by(part_col data_type3);
load data inpath '/hdfs/loc/directory/' into table <staging_table>;
SET hive.exec.dynamic.partition=true;
SET hive.exec.dynamic.partition.mode=nonstrict;
insert into table <main_table>
partition(part_col)
select col1,col2,....part_col from <staging_table>;
Key aspects in approach 2 are:
Making the 'part_col' available as a field in the loading file
In the final insert statement, get 'part_col' as the last field from select clause.
Let's create a table with a partition on a year and month, with a timestamp in the table:
CREATE TABLE `mypart_p`(
`id` bigint,
`open_ts` string
)
PARTITIONED BY (YEAR INT, MONTH INT)
Now I have to alter the table.
ALTER TABLE mypart_p ADD PARTITION (YEAR=2020, MONTH=1)
I had to do that for every year and month, did it in a loop in python. Now lets fill it with data and assign which partition that data belongs to:
INSERT into mypart_p PARTITION (YEAR=2020, MONTH=1)
select id,
open_ts
FROM some_other_table
WHERE substring(open_ts,0,4) = '2020'
AND substring(open_ts,6,2) = '01'
Related
I have text data in following format:
Generated by trjconv : P/L=1/400 t= 0.00000
11214
1P1 aP1 1 80.48 35.36 4.25
2P1 aP1 2 37.45 3.92 3.96
3P2 aP2 3 18.53 -9.69 4.68
4P2 aP2 4 55.39 74.34 4.60
Generated by trjconv : P/L=2/400 t= 0.00000
11214
1P1 aP1 1 80.48 35.36 4.25
2P1 aP1 2 37.45 3.92 3.96
3P2 aP2 3 18.53 -9.69 4.68
4P2 aP2 4 55.39 74.34 4.60
Generated by trjconv : P/L=3/400 t= 0.00000
11214
1P1 aP1 1 80.48 35.36 4.25
2P1 aP1 2 37.45 3.92 3.96
3P2 aP2 3 18.53 -9.69 4.68
4P2 aP2 4 55.39 74.34 4.60
.
.
.
I'm using the sed command to extract the selected lines
sed -n '3~6p' filename > output_file1
The data has around a million lines and one frame of data has around 11217 lines. I want to extract these lines and store it to the new file. With this command I have to do that 11217 times. Can someone show how to use to for loop with sed command? Output should look like this,
output1.txt
1P1 aP1 1 80.48 35.36 4.25
1P1 aP1 1 80.48 35.36 4.25
1P1 aP1 1 80.48 35.36 4.25
output2.txt
2P1 aP1 2 37.45 3.92 3.96
2P1 aP1 2 37.45 3.92 3.96
2P1 aP1 2 37.45 3.92 3.96
output3.txt
3P2 aP2 3 18.53 -9.69 4.68
3P2 aP2 3 18.53 -9.69 4.68
3P2 aP2 3 18.53 -9.69 4.68
.
.
.
output11217.txt
11217P11217 aP11217 11217 18.53 -9.69 4.68
...
.
.
.
Using a loop for that would be an anti-pattern, just do this:
$ cat tst.sh
#!/usr/bin/env bash
awk -v OFS='\t' 'sub(/^ +/,""){print $1, NR, $0}' "${#:--}" |
sort -k1,1 -k2,2n |
cut -f3- |
awk '
$1 != prev {
close(out)
out = "output" (++c) ".txt"
prev = $1
}
{ print > out }
'
$ ./tst.sh file
$ head output*
==> output1.txt <==
1P1 aP1 1 80.48 35.36 4.25
1P1 aP1 1 80.48 35.36 4.25
1P1 aP1 1 80.48 35.36 4.25
==> output2.txt <==
2P1 aP1 2 37.45 3.92 3.96
2P1 aP1 2 37.45 3.92 3.96
2P1 aP1 2 37.45 3.92 3.96
==> output3.txt <==
3P2 aP2 3 18.53 -9.69 4.68
3P2 aP2 3 18.53 -9.69 4.68
3P2 aP2 3 18.53 -9.69 4.68
==> output4.txt <==
4P2 aP2 4 55.39 74.34 4.60
4P2 aP2 4 55.39 74.34 4.60
4P2 aP2 4 55.39 74.34 4.60
If you're not sure what the above is doing just execute each of the commands in order, adding the next command to the pipeline in each step.
This might work for you (GNU utils):
sed '/^\s/!d;s/^\s*//' file |
sort -ns |
uniq -w 8 --group |
csplit -qf output -b %d.txt --suppress - '/^$/' '{*}'
Filter file removing unwanted lines and spaces.
Sort numerically using a stable sort to preserve duplicates position.
Separate groups of lines by a space line.
Split files on space lines.
N.B. Output files are named from 0 not 1.
I have a dataset in this form:
company_name date
0 global_infotech 2019-06-15
1 global_infotech 2020-03-22
2 global_infotech 2020-08-30
3 global_infotech 2018-06-19
4 global_infotech 2018-06-15
5 global_infotech 2018-02-15
6 global_infotech 2018-11-22
7 global_infotech 2019-01-15
8 global_infotech 2018-12-15
9 global_infotech 2019-06-15
10 global_infotech 2018-12-19
11 global_infotech 2019-12-31
12 global_infotech 2019-02-18
13 global_infotech 2018-06-16
14 global_infotech 2019-02-10
15 global_infotech 2019-03-15
16 Qualcom 2019-07-11
17 Qualcom 2018-01-11
18 Qualcom 2018-05-29
19 Qualcom 2018-10-06
20 Qualcom 2018-11-11
21 Qualcom 2019-08-17
22 Qualcom 2019-02-22
23 Qualcom 2019-10-16
24 Qualcom 2018-06-22
25 Qualcom 2018-06-14
26 Qualcom 2018-06-16
27 Syscin 2018-02-10
28 Syscin 2019-02-16
29 Syscin 2018-04-12
30 Syscin 2018-08-22
31 Syscin 2018-09-16
32 Syscin 2019-04-20
33 Syscin 2018-02-28
34 Syscin 2018-01-19
CONSIDERING TODAY'S DATE AS 1st JANUARY 2020, I WANT TO WRITE A CODE TO FIND THE NUMBER OF TIMES EACH COMPANY NAME IS OCCURING IN LAST 3 MONTHS. For example, suppose from 1st Oct 2019 to 1st Jan 2020, golbal_infotech's name is appearing 5 times, then 5 should appear infront of every global_infotech value like:
company_name date appearance_count_last_3_months
0 global_infotech 2019-06-15 5
1 global_infotech 2020-03-22 5
2 global_infotech 2020-08-30 5
3 global_infotech 2018-06-19 5
4 global_infotech 2018-06-15 5
5 global_infotech 2018-02-15 5
6 global_infotech 2018-11-22 5
7 global_infotech 2019-01-15 5
8 global_infotech 2018-12-15 5
9 global_infotech 2019-06-15 5
10 global_infotech 2018-12-19 5
11 global_infotech 2019-12-31 5
12 global_infotech 2019-02-18 5
13 global_infotech 2018-06-16 5
14 global_infotech 2019-02-10 5
15 global_infotech 2019-03-15 5
IIUC:
you can create a custom function:
def getcount(company,month=3,df=df):
df=df.copy()
df['date']=pd.to_datetime(df['date'],format='%Y-%m-%d',errors='coerce')
df=df[df['company_name'].eq(company)]
val=df.groupby(pd.Grouper(key='date',freq=str(month)+'m')).count().max().get(0)
df['appearance_count_last_3_months']=val
return df
getcount('global_infotech')
#OR
getcount('global_infotech',3)
Update:
since you have 92 different companies so you can use for loop:
lst=[]
for x in df['company_name'].unique():
lst.append(getcount(x))
out=pd.concat(lst)
If you print out then you will get your desired output
You can first filter the data for the last 3 months, and then groupby company name and merge back into the original dataframe.
import pandas as pd
from datetime import datetime
from dateutil.relativedelta import relativedelta
# sample data
df = pd.DataFrame({
'company_name': ['global_infotech', 'global_infotech', 'Qualcom','another_company'],
'date': ['2019-02-18', '2021-07-02', '2021-07-01','2019-02-18']
})
df['date'] = pd.to_datetime(df['date'])
# filter for last 3 months
summary = df[df['date']>=datetime.now()-relativedelta(months=3)]
# groupby then aggregate with desired column name
summary = summary.rename(columns={'date':'appearance_count_last_3_months'})
summary = summary.groupby('company_name')
summary = summary.agg('count')
# merge summary back into original df, filling missing values with 0
df = df.merge(summary, left_on='company_name', right_index=True, how='left')
df['appearance_count_last_3_months'] = df['appearance_count_last_3_months'].fillna(0).astype('int')
# result:
df
company_name date appearance_count_last_3_months
0 global_infotech 2019-02-18 1
1 global_infotech 2021-07-02 1
2 Qualcom 2021-07-01 1
3 another_company 2019-02-18 0
I have a dataset that I collected over many days and is indexed by calendar day. Each day has a different number of entries in it. I want to see if the odd days (e.g. day 1, day 3, day 5, etc...) are correlated with the even days (e.g. day 2, day 4, day 6 etc...) and to do this, I have to split my dataset into two.
I can't use day % 2 because I have missing days and weekends in the set that throw it off. I have tried using resample like this:
df_odd = df.resample('2D')
lowest_date = df['date_minus_time'].min()
df_even = df.query('date_minus_time != #lowest_date).resample('2D')
But this insists on aggregating the data by day. I want to keep all the rows so I can perform further operations (e.g. groupby) on the resulting datasets.
How can I create two dataframes, one with all the rows with an "even" date and one with all the rows with an "odd" date with even and odd being relative to first day of my data set?
Here are some example data:
Date var
2018-12-10 1
2018-12-10 0
2018-12-10 1
2018-12-10 0
2018-12-11 1
2018-12-11 1
2018-12-12 0
2018-12-12 1
2018-12-12 1
2018-12-14 1
2018-12-14 0
2018-12-14 1
2018-12-16 1
2018-12-16 1
2018-12-16 1
And the expected output:
df_odd:
Date var
2018-12-10 1
2018-12-10 0
2018-12-10 1
2018-12-10 0
2018-12-12 0
2018-12-12 1
2018-12-12 1
2018-12-16 1
2018-12-16 1
2018-12-16 1
df_even:
Date var
2018-12-11 1
2018-12-11 1
2018-12-14 1
2018-12-14 0
2018-12-14 1
Use pd.Categorical with .codes
num = pd.Categorical(df.Date).codes + 1
df_odd = df[num%2 == 0]
df_even = df[num%2 == 1]
df_odd
Date var
0 2018-12-10 1
1 2018-12-10 0
2 2018-12-10 1
3 2018-12-10 0
6 2018-12-12 0
7 2018-12-12 1
8 2018-12-12 1
12 2018-12-16 1
13 2018-12-16 1
14 2018-12-16 1
df_even
Date var
4 2018-12-11 1
5 2018-12-11 1
9 2018-12-14 1
10 2018-12-14 0
11 2018-12-14 1
I have a dataframe r with 165 counts, I save it in text format onto hdfs with the command below:
scala> r.rdd.saveAsTextFile("top3_text")
here is the list of hdfs files (to save space here, I kept only portion of the list):
[paslechoix#gw03 ~]$ hdfs dfs -ls top3_text/*
-rw-r--r-- 3 paslechoix hdfs 0 2018-03-21 22:46 top3_text/_SUCCESS
-rw-r--r-- 3 paslechoix hdfs 55 2018-03-21 22:46 top3_text/part-00025
-rw-r--r-- 3 paslechoix hdfs 57 2018-03-21 22:46 top3_text/part-00026
-rw-r--r-- 3 paslechoix hdfs 54 2018-03-21 22:46 top3_text/part-00027
-rw-r--r-- 3 paslechoix hdfs 54 2018-03-21 22:46 top3_text/part-00028
-rw-r--r-- 3 paslechoix hdfs 0 2018-03-21 22:46 top3_text/part-00029
-rw-r--r-- 3 paslechoix hdfs 0 2018-03-21 22:46 top3_text/part-00030
-rw-r--r-- 3 paslechoix hdfs 0 2018-03-21 22:46 top3_text/part-00031
-rw-r--r-- 3 paslechoix hdfs 0 2018-03-21 22:46 top3_text/part-00032
-rw-r--r-- 3 paslechoix hdfs 0 2018-03-21 22:46 top3_text/part-00033
-rw-r--r-- 3 paslechoix hdfs 0 2018-03-21 22:46 top3_text/part-00034
-rw-r--r-- 3 paslechoix hdfs 0 2018-03-21 22:46 top3_text/part-00035
-rw-r--r-- 3 paslechoix hdfs 0 2018-03-21 22:46 top3_text/part-00036
-rw-r--r-- 3 paslechoix hdfs 0 2018-03-21 22:46 top3_text/part-00197
-rw-r--r-- 3 paslechoix hdfs 54 2018-03-21 22:46 top3_text/part-00198
-rw-r--r-- 3 paslechoix hdfs 54 2018-03-21 22:46 top3_text/part-00199
[paslechoix#gw03 ~]$ hdfs dfs -cat top3_text/part-00163
[paslechoix#gw03 ~]$ hdfs dfs -cat top3_text/part-00162
[paslechoix#gw03 ~]$ hdfs dfs -cat top3_text/part-00199
[663,30,139.99,1]
[664,30,139.99,2]
[665,30,139.99,3]
This is interesting:
1. What makes the saveAsTextFile generate 200 files?
2. What makes some files are empty while others might contain multiple records?
Thank you.
I found the reason!
It is actually decided by the spark.sql.shuffle.partitions setting, the default value is 200: https://spark.apache.org/docs/1.6.1/sql-programming-guide.html
Reducer number
In Shark, default reducer number is 1 and is controlled
by the property mapred.reduce.tasks. Spark SQL deprecates this
property in favor of spark.sql.shuffle.partitions, whose default value
is 200. Users may customize this property via SET:
SET spark.sql.shuffle.partitions=10;
I am having problems with my Seam application, I'm not sure if it is from using Maven along with Jetty for testing or simply a misconfiguration on my part. The error I am getting is rather simple, when Seam attempts to close the event context, it expects it to be open and it must have already been closed by something else. The problem is what and how to determine what caused that.
Here is the stack trace not that it helps much:
java.lang.IllegalStateException: No active event context
at org.jboss.seam.core.Manager.instance(Manager.java:368)
at org.jboss.seam.servlet.ContextualHttpServletRequest.run(ContextualHttpServletRequest.java:55)
at org.jboss.seam.web.ContextFilter.doFilter(ContextFilter.java:37)
at org.jboss.seam.servlet.SeamFilter$FilterChainImpl.doFilter(SeamFilter.java:69)
at org.jboss.seam.servlet.SeamFilter$FilterChainImpl.doFilter(SeamFilter.java:73)
at org.jboss.seam.web.HotDeployFilter.doFilter(HotDeployFilter.java:53)
at org.jboss.seam.servlet.SeamFilter$FilterChainImpl.doFilter(SeamFilter.java:69)
at org.jboss.seam.web.LoggingFilter.doFilter(LoggingFilter.java:60)
at org.jboss.seam.servlet.SeamFilter$FilterChainImpl.doFilter(SeamFilter.java:69)
at org.jboss.seam.servlet.SeamFilter.doFilter(SeamFilter.java:158)
at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1139)
at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:378)
at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:181)
at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:765)
at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:417)
at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
at org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114)
at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
at org.mortbay.jetty.Server.handle(Server.java:324)
at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:535)
at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:865)
at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:539)
at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:409)
at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:520)
These are all the jars in my WEB-INF/lib folder
-rw-r--r-- 1 walterw staff 54665 2009-03-02 19:28 activation-1.0.2.jar
-rw-r--r-- 1 walterw staff 443432 2009-03-02 19:28 antlr-2.7.6.jar
-rw-r--r-- 1 walterw staff 43033 2009-03-02 19:23 asm-3.1.jar
-rw-r--r-- 1 walterw staff 1604162 2009-03-02 19:28 axis-1.2.1.jar
-rw-r--r-- 1 walterw staff 32071 2009-03-02 19:28 axis-jaxrpc-1.2.1.jar
-rw-r--r-- 1 walterw staff 148230 2009-06-17 22:27 cglib-asm-1.0.jar
-rw-r--r-- 1 walterw staff 188671 2009-03-02 19:28 commons-beanutils-1.7.0.jar
-rw-r--r-- 1 walterw staff 571259 2009-03-02 19:28 commons-collections-3.2.jar
-rw-r--r-- 1 walterw staff 146108 2009-06-14 18:20 commons-digester-1.8.1.jar
-rw-r--r-- 1 walterw staff 71442 2009-03-02 19:28 commons-discovery-0.2.jar
-rw-r--r-- 1 walterw staff 261809 2009-03-02 19:28 commons-lang-2.4.jar
-rw-r--r-- 1 walterw staff 38015 2009-03-02 19:28 commons-logging-1.0.4.jar
-rw-r--r-- 1 walterw staff 26202 2009-03-02 19:28 commons-logging-api-1.0.4.jar
-rw-r--r-- 1 walterw staff 313898 2009-03-02 19:28 dom4j-1.6.1.jar
-rw-r--r-- 1 walterw staff 50583 2009-03-02 19:27 ejb3-persistence-1.0.2.GA.jar
-rw-r--r-- 1 walterw staff 15506 2009-03-14 11:21 FileIO-2009.3.14.jar
-rw-r--r-- 1 walterw staff 170443 2009-03-02 19:23 flickrapi-1.1.jar
-rw-r--r-- 1 walterw staff 279714 2009-03-02 19:27 hibernate-annotations-3.4.0.GA.jar
-rw-r--r-- 1 walterw staff 66993 2009-03-02 19:27 hibernate-commons-annotations-3.1.0.GA.jar
-rw-r--r-- 1 walterw staff 2266769 2009-03-02 19:27 hibernate-core-3.3.0.SP1.jar
-rw-r--r-- 1 walterw staff 119292 2009-04-03 18:41 hibernate-entitymanager-3.4.0.GA.jar
-rw-r--r-- 1 walterw staff 313785 2009-06-17 22:46 hibernate-search-3.1.1.GA.jar
-rw-r--r-- 1 walterw staff 62574 2009-03-02 19:27 hibernate-validator-3.1.0.GA.jar
-rw-r--r-- 1 walterw staff 630486 2009-06-17 22:46 hsqldb-1.8.0.2.jar
-rw-r--r-- 1 walterw staff 552514 2009-03-12 21:54 javassist-3.7.1.GA.jar
-rw-r--r-- 1 walterw staff 131456 2009-03-02 19:23 java-unrar-0.2.jar
-rw-r--r-- 1 walterw staff 134652 2009-06-13 21:46 jboss-el-1.0_02.CR4.jar
-rw-r--r-- 1 walterw staff 288761 2009-06-17 22:48 jboss-envers-1.2.1.GA-hibernate-3.3.jar
-rw-r--r-- 1 walterw staff 25589 2009-06-17 22:48 jboss-logging-log4j-2.1.0.GA.jar
-rw-r--r-- 1 walterw staff 12623 2009-06-17 22:48 jboss-logging-spi-2.1.0.GA.jar
-rw-r--r-- 1 walterw staff 16148 2009-06-13 21:46 jboss-seam-debug-2.1.2.jar
-rw-r--r-- 1 walterw staff 2507 2009-06-13 21:46 jboss-seam-jul-2.1.2.jar
-rw-r--r-- 1 walterw staff 28223 2009-06-13 21:46 jboss-seam-mail-2.1.2.jar
-rw-r--r-- 1 walterw staff 294735 2009-06-13 21:46 jboss-seam-ui-2.1.2.jar
-rw-r--r-- 1 walterw staff 312629 2009-06-14 18:20 jsf-api-1.2-b19.jar
-rw-r--r-- 1 walterw staff 302352 2009-03-02 19:23 jsf-facelets-1.1.15.B1.jar
-rw-r--r-- 1 walterw staff 1122787 2009-06-14 18:20 jsf-impl-1.2-b19.jar
-rw-r--r-- 1 walterw staff 13236 2009-03-02 19:28 jta-1.1.jar
-rw-r--r-- 1 walterw staff 367444 2009-03-02 19:28 log4j-1.2.14.jar
-rw-r--r-- 1 walterw staff 822794 2009-06-17 22:46 lucene-core-2.4.1.jar
-rw-r--r-- 1 walterw staff 1139907 2009-06-13 21:46 org.jboss.seam-jboss-seam-2.1.2.jar
-rw-r--r-- 1 walterw staff 445090 2009-03-08 20:11 quartz-1.6.1.jar
-rw-r--r-- 1 walterw staff 171921 2009-06-14 18:20 richfaces-api-3.3.1.GA.jar
-rw-r--r-- 1 walterw staff 1551810 2009-06-14 18:20 richfaces-impl-3.3.1.GA.jar
-rw-r--r-- 1 walterw staff 4160770 2009-06-14 18:21 richfaces-ui-3.3.1.GA.jar
-rw-r--r-- 1 walterw staff 102493 2009-06-17 22:48 SeamCore-2009.06.17.jar
-rw-r--r-- 1 walterw staff 16591 2009-03-02 19:24 slf4j-api-1.5.0.jar
-rw-r--r-- 1 walterw staff 8880 2009-03-02 19:24 slf4j-log4j12-1.5.0.jar
-rw-r--r-- 1 walterw staff 25814 2009-06-17 22:49 WebContent-2009.06.17.jar
-rw-r--r-- 1 walterw staff 109318 2009-03-02 19:28 xml-apis-1.0.b2.jar
I had a few version conflicts before that actually did cause a problem before, I'm just wondering if this could be one of those problems.
Thanks,
Walter
It doesn't look like a maven issue. I'd check one of the following:
Have you set the seam servlet context listener?
<listener>
<listener-class>org.jboss.seam.servlet.SeamListener</listener-class>
</listener>
Based on this question, have you configured additional context filter?
Thanks for your reply.
It appears to actually be a combination of things:
I instantiate ContextualHttpServletRequest inside every requestInitialized and requestDestroyed (ServletRequestListener) to monitor every Http Request.
I have configured the ContextFilter on every URL other than faces (everything not *.xhtml)
My a4j requests are erroring out? I am using RichFaces and see requests in firebug for /a4j resources. I have Seam and RichFaces configured according to the version I'm using 2.2.0.GA.
This issue is resolved for Jetty, but I am getting these errors in Tomcat. I'm trying to gather what might be different between Jetty and Tomcat. The odd thing is, my ServletRequestListener class functions fine as it logs all Http Requests coming into the application, but the view is not actually served up as it is throwing a No Active Event Context Exception.
EDIT:
The reason this was not working on Tomcat but Jetty was I was not catching an exception thrown by the ContextFilter. I now have my own ContextFilter which attempts to create a context for the current request only if the SeamPhaseListener didn't already do so. In some cases, it still fails, that is why I need to catch the exception. This is a bandaid fix, but it works well in tomcat now. I am guessing it is related to jsessionid in conjunction with resources a4j is requesting.
This works well enough for me.
Walter