We use a two-layer Envoy setup.
[front-end] -> E -> [middleware] -> E -> [backend]
Middleware is supposed to take the sharding key from the HTTP metadata and re-transmit it when talking to the backend.
What we have noticed is that Envoy modifies the HTTP header, which is crashing our service inside gRPC.
E1016 11:19:45.808599731 19 call.cc:912] validate_metadata: {"created":"#1602847185.808584663","description":"Illegal header value","file":"external/com_github_grpc_grpc/src/core/lib/surface/validate_metadata.cc","file_line":44,"offset":56,"raw_bytes":"36 37 36 38 33 61 34 34 36 35 36 35 37 30 34 33 36 66 36 34 36 35 34 31 34 39 33 61 36 35 36 33 36 63 36 39 37 30 37 33 36 35 32 64 37 30 36 63 37 35 36 37 36 39 36 65 a5 '67683a44656570436f646541493a65636c697073652d706c7567696e.'\u0000"}
E1016 11:19:45.808619606 19 call_op_set.h:947] assertion failed: false
Any way to avoid this?
UPDATE:
Seems to be only happening with x- headers.
The problem was actually not related to Envoy in the end. Turns out that gRPC strings are not null terminated.
Related
I am trying to read data from kafka as you can see :
var config = new ConsumerConfig
{
BootstrapServers = ""*******,
GroupId = Guid.NewGuid().ToString(),
AutoOffsetReset = AutoOffsetReset.Earliest
};
MessageParser<AdminIpoChange> parser = new(() => new AdminIpoChange());
using (var consumer = new ConsumerBuilder<Ignore, byte[]>(config).Build())
{
consumer.Subscribe("AdminIpoChange");
while (true)
{
AdminIpoChange item = new AdminIpoChange();
var cr = consumer.Consume();
item = parser.ParseFrom(new ReadOnlySpan<byte>(cr.Message.Value).ToArray());
}
consumer.Close();
}
I am using google protobuf for send and receive data .This code returns this error in parser line:
KafkaConsumer.ConsumeAsync: Protocol message end-group tag did not match expected tag.
Google.Protobuf.InvalidProtocolBufferException: Protocol message end-group tag did not match expected tag.
at Google.Protobuf.ParsingPrimitivesMessages.CheckLastTagWas(ParserInternalState& state, UInt32 expectedTag)
at Google.Protobuf.ParsingPrimitivesMessages.ReadGroup(ParseContext& ctx, Int32 fieldNumber, UnknownFieldSet set)
at Google.Protobuf.UnknownFieldSet.MergeFieldFrom(ParseContext& ctx)
at Google.Protobuf.UnknownFieldSet.MergeFieldFrom(UnknownFieldSet unknownFields, ParseContext& ctx)
at AdminIpoChange.pb::Google.Protobuf.IBufferMessage.InternalMergeFrom(ParseContext& input) in D:\MofidProject\domain\obj\Debug\net6.0\Protos\Rlc\AdminIpoChange.cs:line 213
at Google.Protobuf.ParsingPrimitivesMessages.ReadRawMessage(ParseContext& ctx, IMessage message)
at Google.Protobuf.CodedInputStream.ReadRawMessage(IMessage message)
at AdminIpoChange.MergeFrom(CodedInputStream input) in D:\MofidProject\domain\obj\Debug\net6.0\Protos\Rlc\AdminIpoChange.cs:line 188
at Google.Protobuf.MessageExtensions.MergeFrom(IMessage message, Byte[] data, Boolean discardUnknownFields, ExtensionRegistry registry)
at Google.Protobuf.MessageParser`1.ParseFrom(Byte[] data)
at infrastructure.Queue.Kafka.KafkaConsumer.ConsumeCarefully[T](Func`2 consumeFunc, String topic, String group) in D:\MofidProject\infrastructure\Queue\Kafka\KafkaConsumer.cs:line 168
D:\MofidProject\mts.consumer.plus\bin\Debug\net6.0\mts.consumer.plus.exe (process 15516) exited with code -1001.
To automatically close the console when debugging stops, enable Tools->Options->Debugging->Automatically close the console when debugging stops.'
Updated:
My sample data that comes from Kafka :
- {"SymbolName":"\u0641\u062F\u0631","SymbolIsin":"IRo3pzAZ0002","Date":"1400/12/15","Time":"08:00-12:00","MinPrice":17726,"MaxPrice":21666,"Share":1000,"Show":false,"Operation":0,"Id":"100d8e0b54154e9d902054bff193e875","CreateDateTime":"2022-02-26T09:47:20.0134757+03:30"}
My rlc Model :
syntax = "proto3";
message AdminIpoChange
{
string Id =1;
string SymbolName =2;
string SymbolIsin =3;
string Date =4;
string Time=5;
double MinPrice =6;
double MaxPrice =7;
int32 Share =8;
bool Show =9;
int32 Operation =10;
string CreateDateTime=11;
enum AdminIpoOperation
{
Add = 0;
Edit = 1;
Delete = 2;
}
}
My data in bytes :
7B 22 53 79 6D 62 6F 6C 4E 61 6D 65 22 3A 22 5C 75 30 36 34 31 5C 75 30 36 32 46 5C 75 30
36 33 31 22 2C 22 53 79 6D 62 6F 6C 49 73 69 6E 22 3A 22 49 52 6F 33 70 7A 41 5A 30 30 30
32 22 2C 22 44 61 74 65 22 3A 22 31 34 30 30 2F 31 32 2F 31 35 22 2C 22 54 69 6D 65 22 3A
22 30 38 3A 30 30 2D 31 32 3A 30 30 22 2C 22 4D 69 6E 50 72 69 63 65 22 3A 31 37 37 32 36
2C 22 4D 61 78 50 72 69 63 65 22 3A 32 31 36 36 36 2C 22 53 68 61 72 65 22 3A 31 30 30 30
2C 22 53 68 6F 77 22 3A 66 61 6C 73 65 2C 22 4F 70 65 72 61 74 69 6F 6E 22 3A 30 2C 22 49
64 22 3A 22 31 30 30 64 38 65 30 62 35 34 31 35 34 65 39 64 39 30 32 30 35 34 62 66 66 31
39 33 65 38 37 35 22 2C 22 43 72 65 61 74 65 44 61 74 65 54 69 6D 65 22 3A 22 32 30 32 32
2D 30 32 2D 32 36 54 30 39 3A 34 37 3A 32 30 2E 30 31 33 34 37 35 37 2B 30 33 3A 33 30 22
7D
The data is definitely not protobuf binary; byte 0 starts a group with field number 15; inside this group is:
field 4, string
field 13, fixed32
field 6, varint
field 12, fixed32
field 6, varint
after this (at byte 151), an end-group token is encountered with field number 6
There are many striking things about this:
your schema doesn't use groups (in fact, the mere existence of groups is now hard to find in the docs), so ... none of this looks right
end-group tokens are always required to match the last start-group field number, which it doesn't
fields inside a single level are usually (although as a "should", not a "must") written in numerical order
you have no field 12 or 13 declared
your field 6 is of the wrong type - we expect fixed64 here, but got varint
So: there's no doubt about it: that data is ... not what you expect. It certainly isn't valid protobuf binary. Without knowing how that data is stored, all we can do is guess, but on a hunch: let's try decoding it as UTF8 and see what it looks like:
{"SymbolName":"\u0641\u062F\u0631","SymbolIsin":"IRo3pzAZ0002","Date":"1400/12/15","Time":"08:00-12:00","MinPrice":17726,"MaxPrice":21666,"Share":1000,"Show":false,"Operation":0,"Id":"100d8e0b54154e9d902054bff193e875","CreateDateTime":"2022-02-26T09:47:20.0134757+03:30"}
or (formatted)
{
"SymbolName":"\u0641\u062F\u0631",
"SymbolIsin":"IRo3pzAZ0002",
"Date":"1400/12/15",
"Time":"08:00-12:00",
"MinPrice":17726,
"MaxPrice":21666,
"Share":1000,
"Show":false,
"Operation":0,
"Id":"100d8e0b54154e9d902054bff193e875",
"CreateDateTime":"2022-02-26T09:47:20.0134757+03:30"
}
Oops! You've written the data as JSON, and you're trying to decode it as binary protobuf. Decode it as JSON instead, and you should be fine. If this was written with the protobuf JSON API: decode it with the protobuf JSON API.
The code below is an automated CookieClicker I wrote for experimenting with ActionChains. It's based on a tutorial video, at 9:42. (Link)
When I run this code, the for loop runs down 1000 times but only 1 click happens. Multiple clicks only happen if I remove "#" from the commented line, so I run actions.click(cookie) each time. As for the video, that one extra line of code is not necessary. What can be the cause of that?
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.common.action_chains import ActionChains
s = Service("C:\Program Files (x86)\chromedriver.exe")
driver = webdriver.Chrome(service=s)
driver.maximize_window()
driver.get("https://orteil.dashnet.org/cookieclicker/")
driver.implicitly_wait(5)
cookie=driver.find_element(By.ID,"bigCookie")
cookie_count = driver.find_element(By.ID,"cookies")
actions = ActionChains(driver)
actions.click(cookie)
for i in range(1000):
#actions.click(cookie)
actions.perform()
count=int(cookie_count.text.split(" ")[0])
print(i,count)
driver.quit()
The ActionChains implementation
ActionChains can be used in a chain pattern. When you call methods for actions on the ActionChains object, the actions are stored in a queue in the ActionChains object. When you call perform(), the events are fired in the order they are queued up.
perform()
Performs all stored actions.
Conclusion
perform() would fire the events stored in the queue. In your usecase, the actions.click(cookie) is the event.
Your optimal code block will be:
driver.get("https://orteil.dashnet.org/cookieclicker/")
cookie_count = WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.CSS_SELECTOR, "#cookies")))
cookie = WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "#bigCookie")))
for i in range(100):
ActionChains(driver).click(cookie).perform()
count = cookie_count.text.split(" ")[0]
print(i,count)
driver.quit()
Console Output:
0 0
1 1
2 2
3 3
4 4
5 5
6 6
7 7
8 8
9 9
10 10
11 11
12 12
13 13
14 14
15 15
16 16
17 17
18 18
19 19
20 20
21 21
22 22
23 23
24 24
25 25
26 26
27 27
28 28
29 29
30 30
31 31
32 32
33 33
34 34
35 35
36 36
37 37
38 38
39 39
40 40
41 41
42 42
43 43
44 44
45 45
46 46
47 47
48 48
49 49
50 50
51 51
52 52
53 53
54 54
55 55
56 56
57 57
58 58
59 59
60 60
61 61
62 62
63 63
64 64
65 65
66 66
67 67
68 68
69 69
70 70
71 71
72 72
73 73
74 74
75 75
76 76
77 77
78 78
79 79
80 80
81 81
82 82
83 83
84 84
85 85
86 86
87 87
88 88
89 89
90 90
91 91
92 92
93 93
94 94
95 95
96 96
97 97
98 98
99 99
I assume you are using actions, for the sake of using it, or learning about it, since you could simply call cookie.click() to get the desired result.
Actions are used when you need to perform some "action" to an element other than find it or click on it, i.e. you want to right click, or click and hold, or send a keystroke combination and so on. Check Selenium Actions for more info.
As for using actions to click, you need to also understand that perform gets the composite object (call the build function of Actions) of your actions and execute them. Since your actions are declared outside the for loop, after the first click, the perform() function has no more actions to perform.
TLDR:
Either remove the comment of actions.click(cookie) inside your for loop or use cookie.click() to get the same result without using actions.
for i in range(10):
actions.click(cookie)
actions.perform()
#cookie.click()
count=int(cookie_count.text.split(" ")[0])
print(i,count)
driver.quit()
Colab Notebook of it working
I have the following data in pl-sql:
super stage per
04031 BRICK 43
04031 BRIKF 31
04031 SLAB 27
04031 SSLAB 38
04123 BRICK 59
04123 ROOFF 59
04123 SITE 38
04221 BRICK 56
04221 ROOFF 64
04221 ROOFT 40
04221 SETS 100
04221 SITE 39
04221 SLAB 33
I want to make it so the data switches to have the stages as the header and the percentages as the values like so:
super BRICK BRICKF SLAB SSLAB ROOFF ROOFT SETS SITE
04031 43 31 27 38
04123 59 59 38
04221 56 33 64 40 100 39
Try this method for Pivoting the table
select * from
(select super , stage , per from table)
pivot(per for satge in ('BRICK', 'BRICKF', 'SLAB', 'SSLAB', 'ROOFF', 'ROOFT', 'SETS', 'SITE')
I have two text boxes. The first contains this text just like shown.
I need to remove the first 7 characters of each row then show the edited text in the second box.
The first number is different every time so I can't use this
RawText.Text = Replace(RawText.Text, "1757792", " ")
TextFilter.Text = RawText.Text
because the number changes every row.
Is there a way to have a button remove ALL instances of ANY text 7 characters long?
1757792 02 08 09 10 15 21 22 29 34 40 44 46 47 48 53 56 58 68 69 71
1757793 01 07 16 20 22 25 30 36 38 39 42 48 49 51 58 66 70 72 79 80
1757794 01 02 07 09 10 18 29 32 35 36 48 53 54 56 62 65 68 69 71 73
1757795 01 02 06 09 12 18 23 27 30 35 43 52 57 59 60 61 62 73 74 76
1757796 01 11 13 14 18 19 22 31 34 41 45 46 54 57 61 70 71 72 79 80
1757797 01 08 10 18 19 21 32 41 43 44 45 54 61 62 64 66 68 73 74 80
1757798 02 03 06 09 10 23 27 28 33 36 38 41 49 53 60 61 64 73 74 80
1757799 02 12 16 34 36 44 51 52 55 57 58 59 64 71 73 75 76 78 79 80
1757800 05 11 13 17 18 19 23 24 27 31 34 38 39 45 48 61 67 73 79 80
1757801 17 23 29 31 35 38 43 45 48 51 56 57 60 64 65 66 67 73 77 78
1757802 05 06 11 14 17 20 21 27 28 29 33 41 45 49 58 66 67 73 79 80
1757803 06 07 10 11 12 19 20 21 25 30 33 35 38 42 46 51 65 66 75 80
1757804 06 14 16 19 20 23 32 42 43 44 48 52 62 67 68 69 71 72 74 78
You can use string methods like Substring. If you really want to remove the first 7 you can use String.Substring:
Dim txt2Lines = From l In RawText.Lines
Let index = Math.Min(l.Length, 7)
Select l.Substring(index)
txt2.Lines = txt2Lines.ToArray()
This handles also the case that there are also shorter lines.
Note that it doesn't remove the leading space since that is not part of the first seven characters. You could use l.Substring(index).TrimStart().
Another approach is to search the first space and remove everything before that:
Dim txt2Lines = From l In RawText.Lines
Let index = Math.Max(l.IndexOf(" "), 0)
Select l.Substring(index)
txt2.Lines = txt2Lines.ToArray()
String.IndexOf returns -1 if the substring wasn't found, that's why i've used Math.Max(l.IndexOf(" "), 0). In that case the full line should be taken.
You could use String.Split to split the text at the vbCrLf (line break), then use String.SubString to select the string parter starting at index 8, and there you are.
And as GSerg pointed out, if you would like to replace all 7 digit occurences try this:
Dim ResultString As String
Try
ResultString = Regex.Replace(SubjectString, "\d{7}", "", RegexOptions.Singleline)
Catch ex As ArgumentException
'Syntax error in the regular expression
End Try
I am trying to format the output of the AWK's printf() function. More precisely, I am trying to print a matrix with very long rows and I would like to wrap them and continue on the next line. What I am trying to do is best illustrated using Fortran. Consider the following Fortran statement:
write(*,'(10I5)')(i,i=1,100)
The output would be the integers in the range 1:100 printed in rows of 10 elements.
Is it possible to do the same in AWK. I could do that by offsetting the index and printing to new line with "\n". The question is whether that can be done in an elegant manner as in Fortran.
Thanks,
As suggested in the comments I would like to explain my Fortran code, given as an example above.
(i,i=1,100) ! => is a do loop going from 1 to 100
write(*,'(10I5)') ! => is a formatted write statement
10I5 says print 10 integers and for each integer allocate 5 character slot
The trick is, that when one exceeds the 10 x 5 character slots given by the formatted write, one jumps on the next line. So one doesn't need the trailing "\n".
This may help you
[akshay#localhost tmp]$ cat test.for
implicit none
integer i
write(*,'(10I5)')(i,i=1,100)
end
[akshay#localhost tmp]$ gfortran test.for
[akshay#localhost tmp]$ ./a.out
1 2 3 4 5 6 7 8 9 10
11 12 13 14 15 16 17 18 19 20
21 22 23 24 25 26 27 28 29 30
31 32 33 34 35 36 37 38 39 40
41 42 43 44 45 46 47 48 49 50
51 52 53 54 55 56 57 58 59 60
61 62 63 64 65 66 67 68 69 70
71 72 73 74 75 76 77 78 79 80
81 82 83 84 85 86 87 88 89 90
91 92 93 94 95 96 97 98 99 100
[akshay#localhost tmp]$ awk 'BEGIN{for(i=1;i<=100;i++)printf("%5d%s",i,i%10?"":"\n")}'
1 2 3 4 5 6 7 8 9 10
11 12 13 14 15 16 17 18 19 20
21 22 23 24 25 26 27 28 29 30
31 32 33 34 35 36 37 38 39 40
41 42 43 44 45 46 47 48 49 50
51 52 53 54 55 56 57 58 59 60
61 62 63 64 65 66 67 68 69 70
71 72 73 74 75 76 77 78 79 80
81 82 83 84 85 86 87 88 89 90
91 92 93 94 95 96 97 98 99 100