Custom theme not working any more in Xcode 8 - xcode8

I had a theme I really liked but it's not working any more in Xcode 8. Any tips?
I tried restarting Xcode and reinstalling the theme in Xcode/UserData/FontAndColorThemes but no luck.


You need to change an extension of the file to .xccolortheme (probably in a shell with mv, as changing it in Finder doesnt always work)


They added a new field to each color in Xcode 8 themes - alpha. Now all colors are 4 numbers separated by a space. Use this as a starting point for ObsidianCode theme:
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>DVTConsoleDebuggerInputTextColor</key>
<string>1 1 1 1</string>
<key>DVTConsoleDebuggerInputTextFont</key>
<string>SFMono-Bold - 11.0</string>
<key>DVTConsoleDebuggerOutputTextColor</key>
<string>1 1 1 1</string>
<key>DVTConsoleDebuggerOutputTextFont</key>
<string>Menlo-Regular - 18.0</string>
<key>DVTConsoleDebuggerPromptTextColor</key>
<string>0.255 0.714 0.27 1</string>
<key>DVTConsoleDebuggerPromptTextFont</key>
<string>SFMono-Bold - 11.0</string>
<key>DVTConsoleExectuableInputTextColor</key>
<string>1 1 1 1</string>
<key>DVTConsoleExectuableInputTextFont</key>
<string>Menlo-Regular - 18.0</string>
<key>DVTConsoleExectuableOutputTextColor</key>
<string>1 1 1 1</string>
<key>DVTConsoleExectuableOutputTextFont</key>
<string>SFMono-Bold - 11.0</string>
<key>DVTConsoleTextBackgroundColor</key>
<string>0.118 0.125 0.157 1</string>
<key>DVTConsoleTextInsertionPointColor</key>
<string>1 1 1 1</string>
<key>DVTConsoleTextSelectionColor</key>
<string>0.397 0.397 0.302 1</string>
<key>DVTDebuggerInstructionPointerColor</key>
<string>0.12 0.31 0.34 1</string>
<key>DVTMarkupTextBackgroundColor</key>
<string>0.18856 0.195 0.22444 1</string>
<key>DVTMarkupTextBorderColor</key>
<string>0.253475 0.2594 0.286485 1</string>
<key>DVTMarkupTextCodeFont</key>
<string>SFMono-Regular - 10.0</string>
<key>DVTMarkupTextEmphasisColor</key>
<string>1 1 1 1</string>
<key>DVTMarkupTextEmphasisFont</key>
<string>.AppleSystemUIFontItalic - 10.0</string>
<key>DVTMarkupTextInlineCodeColor</key>
<string>1 1 1 0.7</string>
<key>DVTMarkupTextLinkColor</key>
<string>0.33 0.247124 0.894195 1</string>
<key>DVTMarkupTextLinkFont</key>
<string>.AppleSystemUIFont - 10.0</string>
<key>DVTMarkupTextNormalColor</key>
<string>1 1 1 1</string>
<key>DVTMarkupTextNormalFont</key>
<string>.AppleSystemUIFont - 10.0</string>
<key>DVTMarkupTextOtherHeadingColor</key>
<string>1 1 1 0.5</string>
<key>DVTMarkupTextOtherHeadingFont</key>
<string>.AppleSystemUIFont - 14.0</string>
<key>DVTMarkupTextPrimaryHeadingColor</key>
<string>1 1 1 1</string>
<key>DVTMarkupTextPrimaryHeadingFont</key>
<string>.AppleSystemUIFont - 24.0</string>
<key>DVTMarkupTextSecondaryHeadingColor</key>
<string>1 1 1 1</string>
<key>DVTMarkupTextSecondaryHeadingFont</key>
<string>.AppleSystemUIFont - 18.0</string>
<key>DVTMarkupTextStrongColor</key>
<string>1 1 1 1</string>
<key>DVTMarkupTextStrongFont</key>
<string>.AppleSystemUIFontBold - 10.0</string>
<key>DVTSourceTextBackground</key>
<string>0.118 0.125 0.157 1</string>
<key>DVTSourceTextBlockDimBackgroundColor</key>
<string>0.5 0.5 0.5 1</string>
<key>DVTSourceTextCurrentLineHighlightColor</key>
<string>0.298039 0.298039 0.298039 1</string>
<key>DVTSourceTextInsertionPointColor</key>
<string>1 1 1 1</string>
<key>DVTSourceTextInvisiblesColor</key>
<string>0.5 0.5 0.5 1</string>
<key>DVTSourceTextSelectionColor</key>
<string>0.498039 0.498039 0.498039 1</string>
<key>DVTSourceTextSyntaxColors</key>
<dict>
<key>xcode.syntax.attribute</key>
<string>0.851 0.730 0.398 1</string>
<key>xcode.syntax.character</key>
<string>0.000 0.000 0.000 1</string>
<key>xcode.syntax.comment</key>
<string>0.471 0.506 0.427 1</string>
<key>xcode.syntax.comment.doc</key>
<string>0.471 0.506 0.427 1</string>
<key>xcode.syntax.comment.doc.keyword</key>
<string>0.471 0.506 0.427 1</string>
<key>xcode.syntax.identifier.class</key>
<string>0.365 0.486 0.616 1</string>
<key>xcode.syntax.identifier.class.system</key>
<string>0.365 0.486 0.616 1</string>
<key>xcode.syntax.identifier.constant</key>
<string>0.440 0.560 0.738 1</string>
<key>xcode.syntax.identifier.constant.system</key>
<string>0.365 0.486 0.616 1</string>
<key>xcode.syntax.identifier.function</key>
<string>0.784 0.792 0.816 1</string>
<key>xcode.syntax.identifier.function.system</key>
<string>0.365 0.486 0.616 1</string>
<key>xcode.syntax.identifier.macro</key>
<string>0.549 0.549 0.714 1</string>
<key>xcode.syntax.identifier.macro.system</key>
<string>0.502 0.814 0.795 1</string>
<key>xcode.syntax.identifier.type</key>
<string>0.873 0.752 0.446 1</string>
<key>xcode.syntax.identifier.type.system</key>
<string>0.365 0.486 0.616 1</string>
<key>xcode.syntax.identifier.variable</key>
<string>0.873 0.752 0.446 1</string>
<key>xcode.syntax.identifier.variable.system</key>
<string>0.365 0.486 0.616 1</string>
<key>xcode.syntax.keyword</key>
<string>0.569 0.757 0.333 1</string>
<key>xcode.syntax.number</key>
<string>0.812 0.671 0.000 1</string>
<key>xcode.syntax.plain</key>
<string>0.851 0.851 0.851 1</string>
<key>xcode.syntax.preprocessor</key>
<string>0.549 0.549 0.714 1</string>
<key>xcode.syntax.string</key>
<string>0.910 0.459 0.000 1</string>
<key>xcode.syntax.url</key>
<string>0.100 0.100 1</string>
</dict>
<key>DVTSourceTextSyntaxFonts</key>
<dict>
<key>xcode.syntax.attribute</key>
<string>Menlo-Regular - 12.0</string>
<key>xcode.syntax.character</key>
<string>Menlo-Regular - 12.0</string>
<key>xcode.syntax.comment</key>
<string>Menlo-Regular - 12.0</string>
<key>xcode.syntax.comment.doc</key>
<string>Menlo-Regular - 12.0</string>
<key>xcode.syntax.comment.doc.keyword</key>
<string>Menlo-Regular - 12.0</string>
<key>xcode.syntax.identifier.class</key>
<string>Menlo-Regular - 12.0</string>
<key>xcode.syntax.identifier.class.system</key>
<string>Menlo-Regular - 12.0</string>
<key>xcode.syntax.identifier.constant</key>
<string>Menlo-Regular - 12.0</string>
<key>xcode.syntax.identifier.constant.system</key>
<string>Menlo-Regular - 12.0</string>
<key>xcode.syntax.identifier.function</key>
<string>Menlo-Regular - 12.0</string>
<key>xcode.syntax.identifier.function.system</key>
<string>Menlo-Regular - 12.0</string>
<key>xcode.syntax.identifier.macro</key>
<string>Menlo-Regular - 12.0</string>
<key>xcode.syntax.identifier.macro.system</key>
<string>Menlo-Regular - 12.0</string>
<key>xcode.syntax.identifier.type</key>
<string>Menlo-Regular - 12.0</string>
<key>xcode.syntax.identifier.type.system</key>
<string>Menlo-Regular - 12.0</string>
<key>xcode.syntax.identifier.variable</key>
<string>Menlo-Regular - 12.0</string>
<key>xcode.syntax.identifier.variable.system</key>
<string>Menlo-Regular - 12.0</string>
<key>xcode.syntax.keyword</key>
<string>Menlo-Regular - 12.0</string>
<key>xcode.syntax.number</key>
<string>Menlo-Regular - 12.0</string>
<key>xcode.syntax.plain</key>
<string>Menlo-Regular - 12.0</string>
<key>xcode.syntax.preprocessor</key>
<string>Menlo-Regular - 12.0</string>
<key>xcode.syntax.string</key>
<string>Menlo-Regular - 12.0</string>
<key>xcode.syntax.url</key>
<string>Menlo-Regular - 12.0</string>
</dict>
</dict>
</plist>

Related

awk print lines until next match on the same line

I have the following type of data file:
0.033333 0.000000 0.000000
-46.956 -46.956 -23.678 -23.677 -23.055 -23.054 -22.974 -22.974 -8.033 -8.032
-7.375 -7.356 -7.182 -7.159 -6.695 -6.661 -6.628 -6.598 -4.477 -4.477
-4.470 -4.462 -4.387 -4.380 3.799 3.800 5.939 5.960 6.116 6.117
6.625 6.642 7.648 7.651 7.686 7.687 8.077 8.078 8.123 8.126
8.478 8.497 8.550 8.552 11.625 11.626 12.652 12.653 12.722 12.726
13.860 13.864 14.291 14.293 14.966 15.046 17.063 17.252 18.011 18.015
0.016667 0.000000 0.000000
-46.956 -46.956 -23.677 -23.677 -23.055 -23.054 -22.974 -22.974 -8.037 -8.036
-7.371 -7.361 -7.177 -7.165 -6.686 -6.669 -6.620 -6.605 -4.476 -4.475
-4.471 -4.465 -4.385 -4.382 3.811 3.812 5.942 5.952 6.115 6.115
6.629 6.638 7.651 7.653 7.688 7.689 8.072 8.073 8.122 8.123
8.491 8.501 8.556 8.556 11.612 11.612 12.665 12.665 12.730 12.733
13.835 13.837 14.288 14.289 14.991 15.031 17.132 17.225 18.053 18.055
0.000000 0.000000 0.000000
-46.956 -46.956 -23.677 -23.677 -23.055 -23.055 -22.974 -22.974 -8.038 -8.038
-7.366 -7.366 -7.172 -7.172 -6.678 -6.678 -6.613 -6.613 -4.475 -4.475
-4.469 -4.469 -4.384 -4.384 3.816 3.816 5.946 5.946 6.115 6.115
6.633 6.633 7.653 7.653 7.689 7.689 8.070 8.070 8.122 8.122
8.498 8.498 8.558 8.558 11.607 11.607 12.668 12.668 12.735 12.735
13.827 13.827 14.287 14.287 15.013 15.013 17.186 17.186 18.068 18.068
I need to change this to look like this:
0.033333 0.000000 0.000000 -46.956 -46.956 -23.678 -23.677 -23.055 -23.054 -22.974 -22.974 -8.033 -8.032 -7.375 -7.356 -7.182 -7.159 -6.695 -6.661 -6.628 -6.598 -4.477 -4.477 -4.470 -4.462 -4.387 -4.380 3.799 3.800 5.939 5.960 6.116 6.117 6.625 6.642 7.648 7.651 7.686 7.687 8.077 8.078 8.123 8.126 8.478 8.497 8.550 8.552 11.625 11.626 12.652 12.653 12.722 12.726 13.860 13.864 14.291 14.293 14.966 15.046 17.063 17.252 18.011 18.015
0.016667 0.000000 0.000000 -46.956 -46.956 -23.677 -23.677 -23.055 -23.054 -22.974 -22.974 -8.037 -8.036 -7.371 -7.361 -7.177 -7.165 -6.686 -6.669 -6.620 -6.605 -4.476 -4.475 -4.471 -4.465 -4.385 -4.382 3.811 3.812 5.942 5.952 6.115 6.115 6.629 6.638 7.651 7.653 7.688 7.689 8.072 8.073 8.122 8.123 8.491 8.501 8.556 8.556 11.612 11.612 12.665 12.665 12.730 12.733 13.835 13.837 14.288 14.289 14.991 15.031 17.132 17.225 18.053 18.055
0.000000 0.000000 0.000000 -46.956 -46.956 -23.677 -23.677 -23.055 -23.055 -22.974 -22.974 -8.038 -8.038 -7.366 -7.366 -7.172 -7.172 -6.678 -6.678 -6.613 -6.613 -4.475 -4.475 -4.469 -4.469 -4.384 -4.384 3.816 3.816 5.946 5.946 6.115 6.115 6.633 6.633 7.653 7.653 7.689 7.689 8.070 8.070 8.122 8.122 8.498 8.498 8.558 8.558 11.607 11.607 12.668 12.668 12.735 12.735 13.827 13.827 14.287 14.287 15.013 15.013 17.186 17.186 18.068 18.068
Basically look for the lines with 3 fields only and from there start to remove the line break character until the next line with 3 fields. Also I want to remove all the spaces at the beginning of the line with the 3 fields. Hope this is clearer from the above example.
I have tried the following code:
BEGIN {
ORS=" ";
}
NF==3 {x=NR+6} (NR<=x) {print}
Trouble is that I get a completely different result. I don't know how to add a \n character before the next pattern match. So I get:
0.033333 0.000000 0.000000 -46.956 -46.956 -23.678 -23.677 -23.055 -23.054 -22.974 -22.974 -8.033 -8.032 -7.375 -7.356 -7.182 -7.159 -6.695 -6.661 -6.628 -6.598 -4.477 -4.477 -4.470 -4.462 -4.387 -4.380 3.799 3.800 5.939 5.960 6.116 6.117 6.625 6.642 7.648 7.651 7.686 7.687 8.077 8.078 8.123 8.126 8.478 8.497 8.550 8.552 11.625 11.626 12.652 12.653 12.722 12.726 13.860 13.864 14.291 14.293 14.966 15.046 17.063 17.252 18.011 18.015 0.016667 0.000000 0.000000 -46.956 -46.956 -23.677 -23.677 -23.055 -23.054 -22.974 -22.974 -8.037 -8.036 -7.371 -7.361 -7.177 -7.165 -6.686 -6.669 -6.620 -6.605 -4.476 -4.475 -4.471 -4.465 -4.385 -4.382 3.811 3.812 5.942 5.952 6.115 6.115 6.629 6.638 7.651 7.653 7.688 7.689 8.072 8.073 8.122 8.123 8.491 8.501 8.556 8.556 11.612 11.612 12.665 12.665 12.730 12.733 13.835 13.837 14.288 14.289 14.991 15.031 17.132 17.225 18.053 18.055 0.000000 0.000000 0.000000 -46.956 -46.956 -23.677 -23.677 -23.055 -23.055 -22.974 -22.974 -8.038 -8.038 -7.366 -7.366 -7.172 -7.172 -6.678 -6.678 -6.613 -6.613 -4.475 -4.475 -4.469 -4.469 -4.384 -4.384 3.816 3.816 5.946 5.946 6.115 6.115 6.633 6.633 7.653 7.653 7.689 7.689 8.070 8.070 8.122 8.122 8.498 8.498 8.558 8.558 11.607 11.607 12.668 12.668 12.735 12.735 13.827 13.827 14.287 14.287 15.013 15.013 17.186 17.186 18.068
I also don't know how to get rid of all the space characters on the line with the pattern match.

One awk idea:
awk '
NF==3 { sub(/^[[:space:]]+/,"") # remove leading white space
printf "%s%s",eol,$0 # initially eol="" (undefined)
eol="\n" # next time print this line with a leading "\n" (to close out previous line)
next}
{ printf "%s%s",OFS,$0 } # OP will need to decide if the extra OFS is needed here or can be removed
END { print "" } # terminate last line of output with a "\n"
' file
This generates:
0.033333 0.000000 0.000000 -46.956 -46.956 -23.678 -23.677 -23.055 -23.054 -22.974 -22.974 -8.033 -8.032 -7.375 -7.356 -7.182 -7.159 -6.695 -6.661 -6.628 -6.598 -4.477 -4.477 -4.470 -4.462 -4.387 -4.380 3.799 3.800 5.939 5.960 6.116 6.117 6.625 6.642 7.648 7.651 7.686 7.687 8.077 8.078 8.123 8.126 8.478 8.497 8.550 8.552 11.625 11.626 12.652 12.653 12.722 12.726 13.860 13.864 14.291 14.293 14.966 15.046 17.063 17.252 18.011 18.015
0.016667 0.000000 0.000000 -46.956 -46.956 -23.677 -23.677 -23.055 -23.054 -22.974 -22.974 -8.037 -8.036 -7.371 -7.361 -7.177 -7.165 -6.686 -6.669 -6.620 -6.605 -4.476 -4.475 -4.471 -4.465 -4.385 -4.382 3.811 3.812 5.942 5.952 6.115 6.115 6.629 6.638 7.651 7.653 7.688 7.689 8.072 8.073 8.122 8.123 8.491 8.501 8.556 8.556 11.612 11.612 12.665 12.665 12.730 12.733 13.835 13.837 14.288 14.289 14.991 15.031 17.132 17.225 18.053 18.055
0.000000 0.000000 0.000000 -46.956 -46.956 -23.677 -23.677 -23.055 -23.055 -22.974 -22.974 -8.038 -8.038 -7.366 -7.366 -7.172 -7.172 -6.678 -6.678 -6.613 -6.613 -4.475 -4.475 -4.469 -4.469 -4.384 -4.384 3.816 3.816 5.946 5.946 6.115 6.115 6.633 6.633 7.653 7.653 7.689 7.689 8.070 8.070 8.122 8.122 8.498 8.498 8.558 8.558 11.607 11.607 12.668 12.668 12.735 12.735 13.827 13.827 14.287 14.287 15.013 15.013 17.186 17.186 18.068 18.068

Since you always have 7 lines per record, all you need is this, using GNU awk for multi-char RS and RT:
$ awk -v RS='([^\n]+\n){7}' -v ORS= '{$0=RT; $1=$1} 1' file
0.033333 0.000000 0.000000 -46.956 -46.956 -23.678 -23.677 -23.055 -23.054 -22.974 -22.974 -8.033 -8.032 -7.375 -7.356 -7.182 -7.159 -6.695 -6.661 -6.628 -6.598 -4.477 -4.477 -4.470 -4.462 -4.387 -4.380 3.799 3.800 5.939 5.960 6.116 6.117 6.625 6.642 7.648 7.651 7.686 7.687 8.077 8.078 8.123 8.126 8.478 8.497 8.550 8.552 11.625 11.626 12.652 12.653 12.722 12.726 13.860 13.864 14.291 14.293 14.966 15.046 17.063 17.252 18.011 18.015
0.016667 0.000000 0.000000 -46.956 -46.956 -23.677 -23.677 -23.055 -23.054 -22.974 -22.974 -8.037 -8.036 -7.371 -7.361 -7.177 -7.165 -6.686 -6.669 -6.620 -6.605 -4.476 -4.475 -4.471 -4.465 -4.385 -4.382 3.811 3.812 5.942 5.952 6.115 6.115 6.629 6.638 7.651 7.653 7.688 7.689 8.072 8.073 8.122 8.123 8.491 8.501 8.556 8.556 11.612 11.612 12.665 12.665 12.730 12.733 13.835 13.837 14.288 14.289 14.991 15.031 17.132 17.225 18.053 18.055
0.000000 0.000000 0.000000 -46.956 -46.956 -23.677 -23.677 -23.055 -23.055 -22.974 -22.974 -8.038 -8.038 -7.366 -7.366 -7.172 -7.172 -6.678 -6.678 -6.613 -6.613 -4.475 -4.475 -4.469 -4.469 -4.384 -4.384 3.816 3.816 5.946 5.946 6.115 6.115 6.633 6.633 7.653 7.653 7.689 7.689 8.070 8.070 8.122 8.122 8.498 8.498 8.558 8.558 11.607 11.607 12.668 12.668 12.735 12.735 13.827 13.827 14.287 14.287 15.013 15.013 17.186 17.186 18.068 18.06
or this using any awk:
$ awk '{rec=rec FS $0} !(NR%7){$0=rec; rec=""; $1=$1; print}' file
0.033333 0.000000 0.000000 -46.956 -46.956 -23.678 -23.677 -23.055 -23.054 -22.974 -22.974 -8.033 -8.032 -7.375 -7.356 -7.182 -7.159 -6.695 -6.661 -6.628 -6.598 -4.477 -4.477 -4.470 -4.462 -4.387 -4.380 3.799 3.800 5.939 5.960 6.116 6.117 6.625 6.642 7.648 7.651 7.686 7.687 8.077 8.078 8.123 8.126 8.478 8.497 8.550 8.552 11.625 11.626 12.652 12.653 12.722 12.726 13.860 13.864 14.291 14.293 14.966 15.046 17.063 17.252 18.011 18.015
0.016667 0.000000 0.000000 -46.956 -46.956 -23.677 -23.677 -23.055 -23.054 -22.974 -22.974 -8.037 -8.036 -7.371 -7.361 -7.177 -7.165 -6.686 -6.669 -6.620 -6.605 -4.476 -4.475 -4.471 -4.465 -4.385 -4.382 3.811 3.812 5.942 5.952 6.115 6.115 6.629 6.638 7.651 7.653 7.688 7.689 8.072 8.073 8.122 8.123 8.491 8.501 8.556 8.556 11.612 11.612 12.665 12.665 12.730 12.733 13.835 13.837 14.288 14.289 14.991 15.031 17.132 17.225 18.053 18.055
0.000000 0.000000 0.000000 -46.956 -46.956 -23.677 -23.677 -23.055 -23.055 -22.974 -22.974 -8.038 -8.038 -7.366 -7.366 -7.172 -7.172 -6.678 -6.678 -6.613 -6.613 -4.475 -4.475 -4.469 -4.469 -4.384 -4.384 3.816 3.816 5.946 5.946 6.115 6.115 6.633 6.633 7.653 7.653 7.689 7.689 8.070 8.070 8.122 8.122 8.498 8.498 8.558 8.558 11.607 11.607 12.668 12.668 12.735 12.735 13.827 13.827 14.287 14.287 15.013 15.013 17.186 17.186 18.068 18.06

awk -v ORS= '
NF==3 {
if (NR>1) print "\n"
sub(/^[[:space:]]*/,"")
}
1;
END { print "\n" }
' file
unset default newline for print (OFS=)
when 3-field line detected
print a newline (unless this is first line)
strip leading whitespace
default print (1;) - with no trailing newline
print final newline at the end
This code assumes all lines have leading whitespace (as shown in the sample input), so that no field separator is needed on joined lines.
Your original code is actually not far from working:
awk '
BEGIN { ORS=" " } # or maybe ORS=""
NF==3 {
sub(/^[[:space:]]*/,"") # strip leading whitespace
x = NR+6
}
NR<=x { print }
NR==x { printf "\n" }
' file
An even simpler solution if we know that the 3-field lines always have much more leading whitespace than any other line (e.g. 8 or more):
awk -v RS='[[:space:]]{8,}' 'gsub(/\n/,"")' file
set input record separator to be lots of spaces
strip all embedded newlines
implicit print will append a trailing newline
Note that the first (empty) record is conveniently elided because gsub fails (no newlines removed) and so does not trigger the implicit print.
Another note: This requires a version of awk that supports multi-character RS (e.g. gawk, busybox; but not mawk, original-awk).
Final note: This method, while shorter code, appears to run significantly more slowly (about 10% of the speed of the first version).
For super-slow (about 1% the speed of the first awk version), and if squeezing whitespace is not a problem, there is also the extremely compact:
<file xargs -n63

CBC solver Result - Linear relaxation infeasible

I am trying to solve an LP problem with the CBC solver in Linux.
When I try t solve it, the model is determined linearly infeasible but a solution is found, so I don't understand why this is happening.
This is the solver output:
Welcome to the CBC MILP Solver
Version: 2.10.5
Build Date: Oct 15 2020
command line - cbc tt2.lp solve printing csv solu sol.txt (default strategy 1)
Presolve determined that the problem was infeasible with tolerance of 1e-08
Analysis indicates model infeasible or unbounded
Perturbing problem by 0.001% of 6368.0221 - largest nonzero change 9.6555471e-05 ( 8.8302642e-05%) - largest zero change 6.095558e-05
0 Obj 0 Primal inf 1303.3614 (30) Dual inf 6484.7494 (8)
0 Obj 0 Primal inf 1303.3614 (30) Dual inf 1.437715e+13 (28)
36 Obj -16980 Primal inf 105.38866 (21) Dual inf 7.2658581e+12 (14) w.o. free dual inf (13)
58 Obj -16046.434 Primal inf 0.28949488 (2)
Primal infeasible - objective value -16046.434
PrimalInfeasible objective -16046.43383 - 58 iterations time 0.002
Result - Linear relaxation infeasible
Enumerated nodes: 0
Total iterations: 0
Time (CPU seconds): 0.01
Time (Wallclock Seconds): 0.02
Option for printingOptions changed from normal to csv
Total time (CPU seconds): 0.01 (Wallclock seconds): 0.03
I turned off preprocessing to see if it made any changes but it didn't.
Welcome to the CBC MILP Solver
Version: 2.10.5
Build Date: Oct 15 2020
command line - cbc tt2.lp preprocess off passp 0 solve printing csv solu sol.txt (default strategy 1)
Option for preprocess changed from sos to off
passPresolve was changed from 5 to 0
Perturbing problem by 0.001% of 6368.0221 - largest nonzero change 9.6555471e-05 ( 8.8302642e-05%) - largest zero change 6.095558e-05
0 Obj 0 Primal inf 1303.3614 (30) Dual inf 6484.7494 (8)
0 Obj 0 Primal inf 1303.3614 (30) Dual inf 1.437715e+13 (28)
36 Obj -16980 Primal inf 105.38866 (21) Dual inf 7.2658581e+12 (14) w.o. free dual inf (13)
58 Obj -16046.434 Primal inf 0.28949488 (2)
Primal infeasible - objective value -16046.434
PrimalInfeasible objective -16046.43383 - 58 iterations time 0.002
Result - Linear relaxation infeasible
Enumerated nodes: 0
Total iterations: 0
Time (CPU seconds): 0.00
Time (Wallclock Seconds): 0.02
Option for printingOptions changed from normal to csv
Total time (CPU seconds): 0.00 (Wallclock seconds): 0.03
So the problem is that the Linear Relaxation of the LP problem is infeasible, but I don't know what this means or how to solve it.
This is the LP file in case it helps.
Minimize
1.0 cv_1_1 - 1.0 w_1_1 + 2.0 cv_2_1 - 2.0 w_2_1 + 4.0 cv_1_2 - 4.0 w_1_2 + 8.0 cv_2_2 - 8.0 w_2_2 + 9.0 cv_1_3 - 9.0 w_1_3 + 18.0 cv_2_3 - 18.0 w_2_3 + 16.0 cv_1_4 - 16.0 w_1_4 + 32.0 cv_2_4 - 32.0 w_2_4 + 1 dmv_2_1 + 1 dmv_2_2
Subject To
\Manipulated variables: initial value
R0: mv_1_0 = 150
R1: mv_2_0 = 150
\States: initial value
R2: x_1_0 = 0.03534155
R3: x_2_0 = 0.07947994
R4: x_3_0 = -0.00548606
R5: x_4_0 = -0.28400902
\Manipulated variables: change variables
R6: dmv_1_1 - mv_1_1 + mv_1_0 = 0
R7: dmv_1_2 - mv_1_2 + mv_1_1 = 0
R8: dmv_1_3 - mv_1_3 + mv_1_2 = 0
R9: dmv_1_4 - mv_1_4 + mv_1_3 = 0
R10: dmv_2_1 - mv_2_1 + mv_2_0 = 0
R11: dmv_2_2 - mv_2_2 + mv_2_1 = 0
R12: dmv_2_3 - mv_2_3 + mv_2_2 = 0
R13: dmv_2_4 - mv_2_4 + mv_2_3 = 0
\Controlled variables: upper and lower bounds
R14: cv_1_1 >= 45
R15: cv_1_1 <= 65
R16: cv_1_2 >= 45
R17: cv_1_2 <= 65
R18: cv_1_3 >= 45
R19: cv_1_3 <= 65
R20: cv_1_4 >= 45
R21: cv_1_4 <= 65
R22: cv_2_1 >= 5
R23: cv_2_1 <= 11
R24: cv_2_2 >= 5
R25: cv_2_2 <= 11
R26: cv_2_3 >= 5
R27: cv_2_3 <= 11
R28: cv_2_4 >= 5
R29: cv_2_4 <= 11
\Manipulated variables: ratio change allowed
R30: dmv_1_1 <= 1
R31: dmv_1_1 >= -1
R32: dmv_1_2 <= 1
R33: dmv_1_2 >= -1
R34: dmv_1_3 <= 0.0
R35: dmv_1_3 >= -0.0
R36: dmv_1_4 <= 0.0
R37: dmv_1_4 >= -0.0
R38: dmv_2_1 <= 1
R39: dmv_2_1 >= -1
R40: dmv_2_2 <= 1
R41: dmv_2_2 >= -1
R42: dmv_2_3 <= 0.0
R43: dmv_2_3 >= -0.0
R44: dmv_2_4 <= 0.0
R45: dmv_2_4 >= -0.0
\Manipulated variables: upper and lower bounds
R46: mv_1_1 >= 140
R47: mv_1_1 <= 170
R48: mv_1_2 >= 140
R49: mv_1_2 <= 170
R50: mv_1_3 >= 140
R51: mv_1_3 <= 170
R52: mv_1_4 >= 140
R53: mv_1_4 <= 170
R54: mv_2_1 >= 130
R55: mv_2_1 <= 150
R56: mv_2_2 >= 130
R57: mv_2_2 <= 150
R58: mv_2_3 >= 130
R59: mv_2_3 <= 150
R60: mv_2_4 >= 130
R61: mv_2_4 <= 150
\W: reference trajectories for controlled variables
R62: w_1_1 = 10.0
R63: w_1_2 = 40.0
R64: w_1_3 = 90.0
R65: w_1_4 = 160.0
R66: w_2_1 = 20.0
R67: w_2_2 = 80.0
R68: w_2_3 = 180.0
R69: w_2_4 = 320.0
\Model
\X(t+1) - Gx(t) - Hmv(t) = 0
\cv(t) - Cx(t) = 0
R70: x_1_2 - 1.001 x_1_1 + 0.0001777 x_2_1 + 0.02923 x_3_1 + 0.001065 x_4_1 - 5.049e-06 mv_1_1 + 8.92e-06 mv_2_1 = 0
R71: x_2_2 - 0.001961 x_1_1 - 0.9536 x_2_1 - 0.009859 x_3_1 + 0.08002 x_4_1 - 9.27e-06 mv_1_1 + 0.0001443 mv_2_1 = 0
R72: x_3_2 - 0.02647 x_1_1 + 0.006174 x_2_1 - 0.9386 x_3_1 + 0.07591 x_4_1 - 5.375e-05 mv_1_1 + 0.000216 mv_2_1 = 0
R73: x_4_2 + 0.001008 x_1_1 - 0.2273 x_2_1 - 0.04935 x_3_1 - 0.6928 x_4_1 - 0.000377 mv_1_1 + 0.001151 mv_2_1 = 0
R74: cv_1_1 - 1408.0 x_1_1 + 10.67 x_2_1 + 19.4 x_3_1 - 0.6612 x_4_1 = 0
R75: cv_2_1 - 4.282 x_1_1 - 77.07 x_2_1 - 0.1379 x_3_1 + 2.884 x_4_1 = 0
R76: x_1_3 - 1.001 x_1_2 + 0.0001777 x_2_2 + 0.02923 x_3_2 + 0.001065 x_4_2 - 5.049e-06 mv_1_2 + 8.92e-06 mv_2_2 = 0
R77: x_2_3 - 0.001961 x_1_2 - 0.9536 x_2_2 - 0.009859 x_3_2 + 0.08002 x_4_2 - 9.27e-06 mv_1_2 + 0.0001443 mv_2_2 = 0
R78: x_3_3 - 0.02647 x_1_2 + 0.006174 x_2_2 - 0.9386 x_3_2 + 0.07591 x_4_2 - 5.375e-05 mv_1_2 + 0.000216 mv_2_2 = 0
R79: x_4_3 + 0.001008 x_1_2 - 0.2273 x_2_2 - 0.04935 x_3_2 - 0.6928 x_4_2 - 0.000377 mv_1_2 + 0.001151 mv_2_2 = 0
R80: cv_1_2 - 1408.0 x_1_2 + 10.67 x_2_2 + 19.4 x_3_2 - 0.6612 x_4_2 = 0
R81: cv_2_2 - 4.282 x_1_2 - 77.07 x_2_2 - 0.1379 x_3_2 + 2.884 x_4_2 = 0
R82: x_1_4 - 1.001 x_1_3 + 0.0001777 x_2_3 + 0.02923 x_3_3 + 0.001065 x_4_3 - 5.049e-06 mv_1_3 + 8.92e-06 mv_2_3 = 0
R83: x_2_4 - 0.001961 x_1_3 - 0.9536 x_2_3 - 0.009859 x_3_3 + 0.08002 x_4_3 - 9.27e-06 mv_1_3 + 0.0001443 mv_2_3 = 0
R84: x_3_4 - 0.02647 x_1_3 + 0.006174 x_2_3 - 0.9386 x_3_3 + 0.07591 x_4_3 - 5.375e-05 mv_1_3 + 0.000216 mv_2_3 = 0
R85: x_4_4 + 0.001008 x_1_3 - 0.2273 x_2_3 - 0.04935 x_3_3 - 0.6928 x_4_3 - 0.000377 mv_1_3 + 0.001151 mv_2_3 = 0
R86: cv_1_3 - 1408.0 x_1_3 + 10.67 x_2_3 + 19.4 x_3_3 - 0.6612 x_4_3 = 0
R87: cv_2_3 - 4.282 x_1_3 - 77.07 x_2_3 - 0.1379 x_3_3 + 2.884 x_4_3 = 0
R88: x_1_5 - 1.001 x_1_4 + 0.0001777 x_2_4 + 0.02923 x_3_4 + 0.001065 x_4_4 - 5.049e-06 mv_1_4 + 8.92e-06 mv_2_4 = 0
R89: x_2_5 - 0.001961 x_1_4 - 0.9536 x_2_4 - 0.009859 x_3_4 + 0.08002 x_4_4 - 9.27e-06 mv_1_4 + 0.0001443 mv_2_4 = 0
R90: x_3_5 - 0.02647 x_1_4 + 0.006174 x_2_4 - 0.9386 x_3_4 + 0.07591 x_4_4 - 5.375e-05 mv_1_4 + 0.000216 mv_2_4 = 0
R91: x_4_5 + 0.001008 x_1_4 - 0.2273 x_2_4 - 0.04935 x_3_4 - 0.6928 x_4_4 - 0.000377 mv_1_4 + 0.001151 mv_2_4 = 0
R92: cv_1_4 - 1408.0 x_1_4 + 10.67 x_2_4 + 19.4 x_3_4 - 0.6612 x_4_4 = 0
R93: cv_2_4 - 4.282 x_1_4 - 77.07 x_2_4 - 0.1379 x_3_4 + 2.884 x_4_4 = 0
Bounds
\States: upper and lower bounds
x_1_1 free
x_1_2 free
x_1_3 free
x_1_4 free
x_1_5 free
x_2_1 free
x_2_2 free
x_2_3 free
x_2_4 free
x_2_5 free
x_3_1 free
x_3_2 free
x_3_3 free
x_3_4 free
x_3_5 free
x_4_1 free
x_4_2 free
x_4_3 free
x_4_4 free
x_4_5 free
End

Disclaimer: I somewhat despise those txt-based serialization-formats and this means, that i'm not an expert because i don't use them often.
My guess:
Variables are by default non-negative
Lower-bound: 0.0
Upper-bound: positive infinity
This is a very common design-decision in various solvers and in regards to the LP-file i guess it might be true too (?).
(A quick glimpse through Gurobi, CPLEX, FICO commercial solvers make me think all of those apply this logic -> CoinOR Clp/Cbc probably does the same)
A:
Let's take some other solver and run your file:
infeasible
says: R4: x_3_0 = -0.00548606 is not okay (there might be other source of infeasibility)
B:
Your lp-file contains:
Bounds
\States: upper and lower bounds
x_1_1 free
...
Free variables are [-infinity, +infinity] variables -> not non-negative
Seeing those explicitly here somewhat indicates a potential default as we guess above
C:
x_3_0 does not appear in free-vars
x_3_0 is probably in [0, +infinity)
Obviously incompatible with R4: x_3_0 = -0.00548606
And it's such a simple deduction, that presolving recognizes that

I meet the same error info, and I solve this problem just by increasing the var bound. For example, (0, 1000) -> (0, 10000).

Plot pandas-data from Alpha Vantage

I just installed the alpha_vantage module in addition to pandas, and have gotten the api-key etc. I now want to plot som data regarding a stock.
Written in the module readme (see here) this is how you do it:
from alpha_vantage.timeseries import TimeSeries
import matplotlib.pyplot as plt
ts = TimeSeries(key='YOUR_API_KEY', output_format='pandas')
data, meta_data = ts.get_intraday(symbol='MSFT',interval='1min', outputsize='full')
data['4. close'].plot()
plt.title('Intraday Times Series for the MSFT stock (1 min)')
plt.show()
But when i write just the same in my project whit my own api-key, i get this error:
File "C:\Users\augbi\PycharmProjects\DjangoProsjekt\main\views.py", line 159, in <module>
data['4. close'].plot()
TypeError: tuple indices must be integers or slices, not str
I printed the data, so you can see the format here ( the top line is the result of data.index:
index: <built-in method index of tuple object at 0x0F7CBC28>
( 1. open 2. high 3. low 4. close 5. volume
date
2021-01-04 19:55:00 3179.94 3183.45 3179.94 3183.45 1317.0
2021-01-04 19:50:00 3179.00 3180.00 3178.26 3178.26 851.0
2021-01-04 18:52:00 3178.11 3178.11 3178.11 3178.11 648.0
2021-01-04 18:15:00 3177.00 3177.00 3177.00 3177.00 505.0
2021-01-04 18:09:00 3177.00 3177.00 3177.00 3177.00 224.0
... ... ... ... ... ...
2020-12-22 07:40:00 3212.78 3212.78 3212.78 3212.78 703.0
2020-12-22 07:34:00 3210.00 3210.00 3210.00 3210.00 755.0
2020-12-22 07:27:00 3208.19 3208.19 3208.19 3208.19 510.0
2020-12-22 07:14:00 3204.00 3204.00 3204.00 3204.00 216.0
2020-12-22 07:08:00 3204.00 3204.00 3204.00 3204.00 167.0
My own code is here:
ts2 = TimeSeries(key='ALPA_KEY', output_format='pandas')
data = ts2.get_intraday(symbol='AMZN',interval='1min', outputsize='full')
print("index:", data.index)
print(data)
data['4. close'].plot()
plt.title('Intraday Times Series for the AMZN stock (1 min)')
plt.show()
My wish is to plot the "4. Closing" column, and if possible the corresponding times. This data represents stock-prise for amazon for 1 day.
Thank you very much in advance!

I had an Alpah-Vantage APIKEY, so I used the following code to check. It didn't come back with an error as you asked. When I checked the output format, it was not the normal pandas format, but an extended format.
data
( 1. open 2. high 3. low 4. close 5. volume
date
2021-01-04 19:55:00 3179.94 3183.45 3179.94 3183.45 1317.0
2021-01-04 19:50:00 3179.00 3180.00 3178.26 3178.26 851.0
2021-01-04 18:52:00 3178.11 3178.11 3178.11 3178.11 648.0
2021-01-04 18:15:00 3177.00 3177.00 3177.00 3177.00 505.0
2021-01-04 18:09:00 3177.00 3177.00 3177.00 3177.00 224.0
... ... ... ... ... ...
2020-12-22 07:40:00 3212.78 3212.78 3212.78 3212.78 703.0
2020-12-22 07:34:00 3210.00 3210.00 3210.00 3210.00 755.0
2020-12-22 07:27:00 3208.19 3208.19 3208.19 3208.19 510.0
2020-12-22 07:14:00 3204.00 3204.00 3204.00 3204.00 216.0
2020-12-22 07:08:00 3204.00 3204.00 3204.00 3204.00 167.0
[3440 rows x 5 columns],
{'1. Information': 'Intraday (1min) open, high, low, close prices and volume',
'2. Symbol': 'AMZN',
'3. Last Refreshed': '2021-01-04 19:55:00',
'4. Interval': '1min',
'5. Output Size': 'Full size',
'6. Time Zone': 'US/Eastern'})
From this format, a normal data frame can be obtained with data[0], so a graph can be created. The following is the code to get the graph.
data[0]
1. open 2. high 3. low 4. close 5. volume
date
2021-01-04 19:55:00 3179.94 3183.45 3179.94 3183.45 1317.0
2021-01-04 19:50:00 3179.00 3180.00 3178.26 3178.26 851.0
2021-01-04 18:52:00 3178.11 3178.11 3178.11 3178.11 648.0
2021-01-04 18:15:00 3177.00 3177.00 3177.00 3177.00 505.0
2021-01-04 18:09:00 3177.00 3177.00 3177.00 3177.00 224.0
ts2 = TimeSeries(key=api_key, output_format='pandas')
data = ts2.get_intraday(symbol='AMZN',interval='1min', outputsize='full')
print("index:", data.index)
print(data)
data[0]['4. close'].plot()
plt.title('Intraday Times Series for the AMZN stock (1 min)')
plt.show()

how to split data in 5 letters around one specific letter for each section

I have a data like this
>sp|Q96A73|P33MX_HUMAN Putative monooxygenase p33MONOX OS=Homo sapiens OX=9606 GN=KIAA1191 PE=1 SV=1
RNDDDDTSVCLGTRQCSWFAGCTNRTWNSSAVPLIGLPNTQDYKWVDRNSGLTWSGNDTCLYSCQNQTKGLLYQLFRNLFCSYGLTEAHGKWRCADASITNDKGHDGHRTPTWWLTGSNLTLSVNNSGLFFLCGNGVYKGFPPKWSGRCGLGYLVPSLTRYLTLNASQITNLRSFIHKVTPHR
>sp|P13674|P4HA1_HUMAN Prolyl 4-hydroxylase subunit alpha-1 OS=Homo sapiens OX=9606 GN=P4HA1 PE=1 SV=2
VECCPNCRGTGMQIRIHQIGPGMVQQIQSVCMECQGHGERISPKDRCKSCNGRKIVREKKILEVHIDKGMKDGQKITFHGEGDQEPGLEPGDIIIVLDQKDHAVFTRRGEDLFMCMDIQLVEALCGFQKPISTLDNRTIVITSHPGQIVKHGDIKCVLNEGMPIYRRPYEKGRLIIEFKVNFPENGFLSPDKLSLLEKLLPERKEVEE
>sp|Q7Z4N8|P4HA3_HUMAN Prolyl 4-hydroxylase subunit alpha-3 OS=Homo sapiens OX=9606 GN=P4HA3 PE=1 SV=1
MTEQMTLRGTLKGHNGWVTQIATTPQFPDMILSASRDKTIIMWKLTRDETNYGIPQRALRGHSHFVSDVVISSDGQFALSGSWDGTLRLWDLTTGTTTRRFVGHTKDVLSVAFSSDNRQIVSGSRDKTIKLWNTLGVCKYTVQDESHSEWVSCVRFSPNSSNPIIVSCGWDKLVKVWNLANCKLK
>sp|P04637|P53_HUMAN Cellular tumor antigen p53 OS=Homo sapiens OX=9606 GN=TP53 PE=1 SV=4
IQVVSRCRLRHTEVLPAEEENDSLGADGTHGAGAMESAAGVLIKLFCVHTKALQDVQIRFQPQL
>sp|P10144|GRAB_HUMAN Granzyme B OS=Homo sapiens OX=9606 GN=GZMB PE=1 SV=2
MQPILLLLAFLLLPRADAGEIIGGHEAKPHSRPYMAYLMIWDQKSLKRCGGFLIRDDFVLTAAHCWGSSINVTLGAHNIKEQEPTQQFIPVKRPIPHPAYNPKNFSNDIMLLQLERKAKRTRAVQPLRLPSNKAQVKPGQTCSVAGWGQTAPLGKHSHTLQEVKMTVQEDRKCES
>sp|Q9UHX1|PUF60_HUMAN Poly(U)-binding-splicing factor PUF60 OS=Homo sapiens OX=9606 GN=PUF60 PE=1 SV=1
MGKDYYQTLGLARGASDEEIKRAYRRQALRYHPDKNKEPGAEEKFKEIAEAYDVLSDPRKREIFDRYGEEGLKGSGPSGGSGGGANGTSFSYTFHGDPHAMFAEFFGGRNPFDTFFGQRNGEEGMDIDDPFSGFPMGMGGFTNVNFGRSRSAQEPARKKQDPPVTHDLRVSLEEIYSGCTKKMKISHK
>sp|Q06416|P5F1B_HUMAN Putative POU domain, class 5, transcription factor 1B OS=Homo sapiens OX=9606 GN=POU5F1B PE=5 SV=2
IVVKGHSTCLSEGALSPDGTVLATASHDGYVKFWQIYIEGQDEPRCLHEWKPHDGRPLSCLLFCDNHKKQDPDVPFWRFLITGADQNRELKMWCTVSWTCLQTIRFSPDIFSSVSVPPSLKVCLDLSAEYLILSDVQRKVLYVMELLQNQEEGHACFSSISEFLLTHPVLSFGIQVVSRCRLRHTEVLPAEEENDSLGADGTHGAGAMESAAGVLIKLFCVHTKALQDVQIRFQPQLNPDVVAPLPTHTAHEDFTFGESRPELGSEGLGSAAHGSQPDLRRIVELPAPADFLSLSSETKPKLMTPDAFMTPSASLQQITASPSSSSSGSSSSSSSSSSSLTAVSAMSSTSAVDPSLTRPPEELTLSPKLQLDGSLTMSSSGSLQASPRGLLPGLLPAPADKLTPKGPGQVPTATSALSLELQEVEP
>sp|O14683|P5I11_HUMAN Tumor protein p53-inducible protein 11 OS=Homo sapiens OX=9606 GN=TP53I11 PE=1 SV=2
MIHNYMEHLERTKLHQLSGSDQLESTAHSRIRKERPISLGIFPLPAGDGLLTPDAQKGGETPGSEQWKFQELSQPRSHTSLKVSNSPEPQKAVEQEDELSDVSQGGSKATTPASTANSDVATIPTDTPLKEENEGFVKVTDAPNKSEISKHIEVQVAQETRNVSTGSAENEEKSEVQAIIESTPELDMDKDLSGYKGSSTPTKGIENKAFDRNTESLFEELSSAGSGLIGDVDEGADLLGMGREVENLILENTQLLETKNALNIVKNDLIAKVDELTCEKDVLQGELEAVKQAKLKLEEKNRELEEELRKARAEAEDARQKAKDDDDSDIPTAQRKRFTRVEMARVLMERNQYKERLMELQEAVRWTEMIRASRENPAMQEKKRSSIWQFFSRLFSSSSNTTKKPEPPVNLKYNAPTSHVTPSVK
I am trying to find 5 letter left and 5 letters right to each F for each section and then calculate the number of E or D in each of them
a representative output looks like below
>sp|Q96A73|P33MX_HUMAN Putative monooxygenase p33MONOX OS=Homo sapiens OX=9606 GN=KIAA1191 PE=1 SV=1
RQCSWFAGCTN 0 0
LLYQLFRNLFC 0 0
LFRNLFCSYGL 0 0
NNSGLFFLCGN 0 0
NSGLFFLCGNG 0 0
GVYKGFPPKWS 0 0
TNLRSFIHKVT 0 0
>sp|P13674|P4HA1_HUMAN Prolyl 4-hydroxylase subunit alpha-1 OS=Homo sapiens OX=9606 GN=P4HA1 PE=1 SV=2
GQKITFHGEGD 1 1
KDHAVFTRRGE 1 1
RGEDLFMCMDI 1 2
EALCGFQKPIS 1 0
RLIIEFKVNFP 1 0
EFKVNFPENGF 2 0
FPENGFLSPDK 1 0
>sp|Q7Z4N8|P4HA3_HUMAN Prolyl 4-hydroxylase subunit alpha-3 OS=Homo sapiens OX=9606 GN=P4HA3 PE=1 SV=1
ATTPQFPDMIL 0 1
RGHSHFVSDVV 0 1
SSDGQFALSGS 0 1
TTTRRFVGHTK 0 0
VLSVAFSSDNR 0 1
VSCVRFSPNSS 0 0
>sp|P04637|P53_HUMAN Cellular tumor antigen p53 OS=Homo sapiens OX=9606 GN=TP53 PE=1 SV=4
VLIKLFCVHTK 0 0
DVQIRFQPQL 0 1
>sp|P10144|GRAB_HUMAN Granzyme B OS=Homo sapiens OX=9606 GN=GZMB PE=1 SV=2
LLLLAFLLLPR 0 0
KRCGGFLIRDD 0 2
LIRDDFVLTAA 0 2
EPTQQFIPVKR 1 0
YNPKNFSNDIM 0 1
>sp|Q9UHX1|PUF60_HUMAN Poly(U)-binding-splicing factor PUF60 OS=Homo sapiens OX=9606 GN=PUF60 PE=1 SV=1
GAEEKFKEIAE 4 0
RKREIFDRYGE 2 1
ANGTSFSYTFH 0 0
SFSYTFHGDPH 0 1
DPHAMFAEFFG 0 1
AMFAEFFGGRN 1 0
MFAEFFGGRNP 1 0
GGRNPFDTFFG 0 1
NPFDTFFGQRN 0 1
PFDTFFGQRNG 0 1
DIDDPFSGFPM 0 3
DPFSGFPMGMG 0 1
MGMGGFTNVNF 0 0
FTNVNFGRSRS 0 0
>sp|Q06416|P5F1B_HUMAN Putative POU domain, class 5, transcription factor 1B OS=Homo sapiens OX=9606 GN=POU5F1B PE=5 SV=2
DGYVKFWQIYI 0 1
LSCLLFCDNHK 0 1
DPDVPFWRFLI 0 2
VPFWRFLITGA 0 0
LQTIRFSPDIF 0 1
FSPDIFSSVSV 0 1
EGHACFSSISE 0 0
SSISEFLLTHP 1 0
HPVLSFGIQVV 0 0
VLIKLFCVHTK 0 0
DVQIRFQPQLN 0 1
TAHEDFTFGES 2 1
HEDFTFGESRP 2 1
PAPADFLSLSS 0 1
MTPDAFMTPSA 0 1
>sp|O14683|P5I11_HUMAN Tumor protein p53-inducible protein 11 OS=Homo sapiens OX=9606 GN=TP53I11 PE=1 SV=2
ISLGIFPLPAG 0 0
SEQWKFQELSQ 2 0
EENEGFVKVTD 3 1
IENKAFDRNTE 2 1
NTESLFEELSS 3 0
AQRKRFTRVEM 1 0
SSIWQFFSRLF 0 0
SIWQFFSRLFS 0 0
FFSRLFSSSSN 0 0
at the begging I thought of finding the 5 letters left and right to F.
but I could not figure it out how to do that

In awk:
$ awk '
NR%2 {print; next } # print every odd record
{ # the even records are processed
while(match($0,/.{5}F.{0,5}/)) { # get 5 before and upto 5 after F
# 5 before F ^^^ ^^^ 0-5 chars after F
# change to /.{0,5}F.{0,5}/ if needed
print s=substr($0,RSTART,RLENGTH), # print match
gsub(/E/,"E",s), # count of Es
gsub(/D/,"D",s) # count of Ds
$0=substr($0,RSTART+1) # shorten the search string
}
}' file
Some output:
>sp|Q96A73|P33MX_HUMAN Putative monooxygenase p33MONOX OS=Homo sapiens OX=9606 GN=KIAA1191 PE=1 SV=1
RQCSWFAGCTN 0 0
LLYQLFRNLFC 0 0 # notice another F in the 5+F+5 window
LFRNLFCSYGL 0 0 # .. getting handled
NNSGLFFLCGN 0 0
NSGLFFLCGNG 0 0
GVYKGFPPKWS 0 0
TNLRSFIHKVT 0 0
...
>sp|P04637|P53_HUMAN Cellular tumor antigen p53 OS=Homo sapiens OX=9606 GN=TP53 PE=1 SV=4
VLIKLFCVHTK 0 0
DVQIRFQPQL 0 1 # ...F{0,5}

Check out this Perl solution
perl -lne ' if (/^[^>]/) { while(/(?<=(\w{5}))F(?=(\w{5}))/g) { $x="$1F$2";$e=()=$x=~/E/g; $d=()=$x=~/D/g; print "$x $e $d" } } else { print }' 5letter.txt
EDIT1:
DVQIRFQPQL 0 1 # Edge case
For accommodating the edge case as mentioned by OP in the comments - the string to the right of F can be less than 5 letters if it is at the end of the line
perl -lne ' if (/^[^>]/) { while(/(?<=(.{5}))F(?=(.{0,5}))/g) { $x="$1F$2";$e=()=$x=~/E/g; $d=()=$x=~/D/g; print "$x $e $d" } } else { print }' 5letter.txt
For each line that doesn't start with >, use positive lookbehind to match 5 letter \w to the left of F and positive lookahead to match 5 letter \w to the right of F. Using while loop and /g on the match operator scan the line and for each match copy the $1F$2 into variable $x. Use list context and count the occurrences of E and D. Print the result finally.
with given inputs
$ perl -lne ' if (/^[^>]/) { while(/(?<=(.{5}))F(?=(.{0,5}))/g) { $x="$1F$2";$e=()=$x=~/E/g; $d=()=$x=~/D/g; print "$x $e $d" } } else { print }' 5letter.txt
>sp|Q96A73|P33MX_HUMAN Putative monooxygenase p33MONOX OS=Homo sapiens OX=9606 GN=KIAA1191 PE=1 SV=1
RQCSWFAGCTN 0 0
LLYQLFRNLFC 0 0
LFRNLFCSYGL 0 0
NNSGLFFLCGN 0 0
NSGLFFLCGNG 0 0
GVYKGFPPKWS 0 0
TNLRSFIHKVT 0 0
>sp|P13674|P4HA1_HUMAN Prolyl 4-hydroxylase subunit alpha-1 OS=Homo sapiens OX=9606 GN=P4HA1 PE=1 SV=2
GQKITFHGEGD 1 1
KDHAVFTRRGE 1 1
RGEDLFMCMDI 1 2
EALCGFQKPIS 1 0
RLIIEFKVNFP 1 0
EFKVNFPENGF 2 0
FPENGFLSPDK 1 1
>sp|Q7Z4N8|P4HA3_HUMAN Prolyl 4-hydroxylase subunit alpha-3 OS=Homo sapiens OX=9606 GN=P4HA3 PE=1 SV=1
ATTPQFPDMIL 0 1
RGHSHFVSDVV 0 1
SSDGQFALSGS 0 1
TTTRRFVGHTK 0 0
VLSVAFSSDNR 0 1
VSCVRFSPNSS 0 0
>sp|P04637|P53_HUMAN Cellular tumor antigen p53 OS=Homo sapiens OX=9606 GN=TP53 PE=1 SV=4
VLIKLFCVHTK 0 0
DVQIRFQPQL 0 1
>sp|P10144|GRAB_HUMAN Granzyme B OS=Homo sapiens OX=9606 GN=GZMB PE=1 SV=2
LLLLAFLLLPR 0 0
KRCGGFLIRDD 0 2
LIRDDFVLTAA 0 2
EPTQQFIPVKR 1 0
YNPKNFSNDIM 0 1
>sp|Q9UHX1|PUF60_HUMAN Poly(U)-binding-splicing factor PUF60 OS=Homo sapiens OX=9606 GN=PUF60 PE=1 SV=1
GAEEKFKEIAE 4 0
RKREIFDRYGE 2 1
ANGTSFSYTFH 0 0
SFSYTFHGDPH 0 1
DPHAMFAEFFG 1 1
AMFAEFFGGRN 1 0
MFAEFFGGRNP 1 0
GGRNPFDTFFG 0 1
NPFDTFFGQRN 0 1
PFDTFFGQRNG 0 1
DIDDPFSGFPM 0 3
DPFSGFPMGMG 0 1
MGMGGFTNVNF 0 0
FTNVNFGRSRS 0 0
>sp|Q06416|P5F1B_HUMAN Putative POU domain, class 5, transcription factor 1B OS=Homo sapiens OX=9606 GN=POU5F1B PE=5 SV=2
DGYVKFWQIYI 0 1
LSCLLFCDNHK 0 1
DPDVPFWRFLI 0 2
VPFWRFLITGA 0 0
LQTIRFSPDIF 0 1
FSPDIFSSVSV 0 1
EGHACFSSISE 2 0
SSISEFLLTHP 1 0
HPVLSFGIQVV 0 0
VLIKLFCVHTK 0 0
DVQIRFQPQLN 0 1
TAHEDFTFGES 2 1
HEDFTFGESRP 2 1
PAPADFLSLSS 0 1
MTPDAFMTPSA 0 1
>sp|O14683|P5I11_HUMAN Tumor protein p53-inducible protein 11 OS=Homo sapiens OX=9606 GN=TP53I11 PE=1 SV=2
ISLGIFPLPAG 0 0
SEQWKFQELSQ 2 0
EENEGFVKVTD 3 1
IENKAFDRNTE 2 1
NTESLFEELSS 3 0
AQRKRFTRVEM 1 0
SSIWQFFSRLF 0 0
SIWQFFSRLFS 0 0
FFSRLFSSSSN 0 0
$
PS:
$ echo "RNDDDDTSVCLGTRQCSWFAGCTNRTWNSSAVPLIGLPNTQDYKWVDRNSGLTWSGNDTCLYSCQNQTKGLLYQLFRNLFCSYGLTEAHGKWRCADASITNDKGHDGHRTPTWWLTGSNLTLSVNNSGLFFLCGNGVYKGFPPKWSGRCGLGYLVPSLT
RYLTLNASQITNLRSFIHKVTPHR" | perl -ne ' if (/^[^>]/) { $y=$_;while(/(.{10})/g) { $x=$1; $c++ for($x=~/F/g) ; print "$x $c\n"; $c=0 } } '
RNDDDDTSVC
LGTRQCSWFA 1
GCTNRTWNSS 0
AVPLIGLPNT 0
QDYKWVDRNS 0
GLTWSGNDTC 0
LYSCQNQTKG 0
LLYQLFRNLF 2
CSYGLTEAHG 0
KWRCADASIT 0
NDKGHDGHRT 0
PTWWLTGSNL 0
TLSVNNSGLF 1
FLCGNGVYKG 1
FPPKWSGRCG 1
LGYLVPSLT 0
RYLTLNASQI 0
TNLRSFIHKV 1
$

With any awk in any shell on any UNIX box:
$ cat tst.awk
/^>/ { print; next }
{
fpos = 0
while ( match(substr($0,fpos+1),/F/) ) {
fpos += RSTART
str = substr($0,fpos-5,11)
print str, gsub(/E/,"&",str), gsub(/D/,"&",str)
}
}
.
$ awk -f tst.awk file
>sp|Q96A73|P33MX_HUMAN Putative monooxygenase p33MONOX OS=Homo sapiens OX=9606 GN=KIAA1191 PE=1 SV=1
RQCSWFAGCTN 0 0
LLYQLFRNLFC 0 0
LFRNLFCSYGL 0 0
NNSGLFFLCGN 0 0
NSGLFFLCGNG 0 0
GVYKGFPPKWS 0 0
TNLRSFIHKVT 0 0
>sp|P13674|P4HA1_HUMAN Prolyl 4-hydroxylase subunit alpha-1 OS=Homo sapiens OX=9606 GN=P4HA1 PE=1 SV=2
GQKITFHGEGD 1 1
KDHAVFTRRGE 1 1
RGEDLFMCMDI 1 2
EALCGFQKPIS 1 0
RLIIEFKVNFP 1 0
EFKVNFPENGF 2 0
FPENGFLSPDK 1 1
>sp|Q7Z4N8|P4HA3_HUMAN Prolyl 4-hydroxylase subunit alpha-3 OS=Homo sapiens OX=9606 GN=P4HA3 PE=1 SV=1
ATTPQFPDMIL 0 1
RGHSHFVSDVV 0 1
SSDGQFALSGS 0 1
TTTRRFVGHTK 0 0
VLSVAFSSDNR 0 1
VSCVRFSPNSS 0 0
>sp|P04637|P53_HUMAN Cellular tumor antigen p53 OS=Homo sapiens OX=9606 GN=TP53 PE=1 SV=4
VLIKLFCVHTK 0 0
DVQIRFQPQL 0 1
>sp|P10144|GRAB_HUMAN Granzyme B OS=Homo sapiens OX=9606 GN=GZMB PE=1 SV=2
LLLLAFLLLPR 0 0
KRCGGFLIRDD 0 2
LIRDDFVLTAA 0 2
EPTQQFIPVKR 1 0
YNPKNFSNDIM 0 1
>sp|Q9UHX1|PUF60_HUMAN Poly(U)-binding-splicing factor PUF60 OS=Homo sapiens OX=9606 GN=PUF60 PE=1 SV=1
GAEEKFKEIAE 4 0
RKREIFDRYGE 2 1
ANGTSFSYTFH 0 0
SFSYTFHGDPH 0 1
DPHAMFAEFFG 1 1
AMFAEFFGGRN 1 0
MFAEFFGGRNP 1 0
GGRNPFDTFFG 0 1
NPFDTFFGQRN 0 1
PFDTFFGQRNG 0 1
DIDDPFSGFPM 0 3
DPFSGFPMGMG 0 1
MGMGGFTNVNF 0 0
FTNVNFGRSRS 0 0
>sp|Q06416|P5F1B_HUMAN Putative POU domain, class 5, transcription factor 1B OS=Homo sapiens OX=9606 GN=POU5F1B PE=5 SV=2
DGYVKFWQIYI 0 1
LSCLLFCDNHK 0 1
DPDVPFWRFLI 0 2
VPFWRFLITGA 0 0
LQTIRFSPDIF 0 1
FSPDIFSSVSV 0 1
EGHACFSSISE 2 0
SSISEFLLTHP 1 0
HPVLSFGIQVV 0 0
VLIKLFCVHTK 0 0
DVQIRFQPQLN 0 1
TAHEDFTFGES 2 1
HEDFTFGESRP 2 1
PAPADFLSLSS 0 1
MTPDAFMTPSA 0 1
>sp|O14683|P5I11_HUMAN Tumor protein p53-inducible protein 11 OS=Homo sapiens OX=9606 GN=TP53I11 PE=1 SV=2
ISLGIFPLPAG 0 0
SEQWKFQELSQ 2 0
EENEGFVKVTD 3 1
IENKAFDRNTE 2 1
NTESLFEELSS 3 0
AQRKRFTRVEM 1 0
SSIWQFFSRLF 0 0
SIWQFFSRLFS 0 0
FFSRLFSSSSN 0 0

This is an adaptation of the excellent solution of James Brown. The adaptation makes it POSIX compliant, and also corrects for the single case where two F values are less than 5 characters apart. Example:
...RQCSWFAGFCTNRQS...
^ ^
The first window should detect RQCSWFAGFCT while the second window should detect SWFAGFCTNRQ. In the proposed solution, it will only detect AGFCTNRQ, or it might not detect the second F at all. (depending on the usage of .{0,5}F.{0,5} or .{5}F.{0,5} as a regex.
awk '
NR%2 {print; next } # print every odd record
{ # the even records are processed
seq=$0; l=lseq=length($0)
while(match(seq,/F/)) { # find `F`
n = l - lseq + RSTART # get position in $0
print s=substr($0,n-5,(n<6?n+5:11)), # print match and
# correct if F is in the first 5
gsub(/E/,"E",s), # count of Es
gsub(/D/,"D",s) # count of Ds
seq=substr(seq,RSTART+1) # shorten the search string
lseq=lseq-RSTART
}
}' file.fasta
You might be also interested in BioAwk, it is an adapted version of awk which is tuned to process FASTA files.
This gives the result:
bioawk -c fastx '{ print ">" $name }
{ tmp=$seq; l=ltmp=length($seq)
while(match(tmp,/F/)) {
n=l-ltmp+RSTART
print s=substr($seq,n,(n<6?n+5:11)),
gsub(/E/,"E",s),
gsub(/D/,"D",s)
tmp=substr(tmp,RSTART+1)
ltmp=ltmp-RSTART
}}' file.fasta
Here $name is the sequence name (everything after >), and $seq is the complete sequence, even if you have a sequence spanning multiple lines.
Note: BioAwk is based on Brian Kernighan's awk which is documented in "The AWK Programming Language",
by Al Aho, Brian Kernighan, and Peter Weinberger
(Addison-Wesley, 1988, ISBN 0-201-07981-X)
. I'm not sure if this version is compatible with POSIX.

Add column to pandas pivot table based on complex condition

my pivot table looks like this:
In [285]: piv
Out[285]:
K 118.5 119.0 119.5 120.0 120.5
Expiry
2018-01-12 0.050842 0.050842 0.050842 0.050842 0.050842
2018-01-19 0.039526 0.039526 0.039526 0.039526 0.039526
2018-01-26 0.039196 0.039196 0.039196 0.039196 0.039196
2018-02-02 0.039991 0.039991 0.039991 0.039991 0.039991
2018-02-23 0.040005 0.040005 0.040005 0.040005 0.040005
2018-03-23 0.041025 0.041000 0.040872 0.040623 0.040398
and df2 looks like this:
In [290]: df2
Out[290]:
F Symbol
Expiry
2018-03-20 12:00:00 123.000000 ZN MAR 18
2018-06-20 12:00:00 122.609375 ZN JUN 18
I am looking to add piv['F'] based on the following:
piv.index.month < df2.index.month
so the result should looks like this:
K F 118.5 119.0 119.5 120.0 120.5
Expiry
2018-01-19 123.000000 0.039526 0.039526 0.039526 0.039526 0.039526
2018-01-26 123.000000 0.039196 0.039196 0.039196 0.039196 0.039196
2018-02-02 123.000000 0.039991 0.039991 0.039991 0.039991 0.039991
2018-02-23 123.000000 0.040005 0.040005 0.040005 0.040005 0.040005
2018-03-23 123.609375 0.041025 0.041000 0.040872 0.040623 0.040398
would help will be much appreciated.

reindex + backfill
df.index=pd.to_datetime(df.index)
df1.index=pd.to_datetime(df1.index)
df['F']=df1.reindex(df.index,method='backfill').F.values
df
Out[164]:
118.5 119.0 119.5 120.0 120.5 F
2018-01-12 0.050842 0.050842 0.050842 0.050842 0.050842 123.000000
2018-01-19 0.039526 0.039526 0.039526 0.039526 0.039526 123.000000
2018-01-26 0.039196 0.039196 0.039196 0.039196 0.039196 123.000000
2018-02-02 0.039991 0.039991 0.039991 0.039991 0.039991 123.000000
2018-02-23 0.040005 0.040005 0.040005 0.040005 0.040005 123.000000
2018-03-23 0.041025 0.041000 0.040872 0.040623 0.040398 122.609375

You want to use pd.merge_asof with direction='forward' and make sure to merge on the indices.
pd.merge_asof(
piv, df2[['F']],
left_index=True,
right_index=True,
direction='forward'
)
118.5 119.0 119.5 120.0 120.5 F
Expiry
2018-01-12 0.050842 0.050842 0.050842 0.050842 0.050842 123.000000
2018-01-19 0.039526 0.039526 0.039526 0.039526 0.039526 123.000000
2018-01-26 0.039196 0.039196 0.039196 0.039196 0.039196 123.000000
2018-02-02 0.039991 0.039991 0.039991 0.039991 0.039991 123.000000
2018-02-23 0.040005 0.040005 0.040005 0.040005 0.040005 123.000000
2018-03-23 0.041025 0.041000 0.040872 0.040623 0.040398 122.609375
And if you want 'F' in front:
pd.merge_asof(
piv, df2[['F']],
left_index=True,
right_index=True,
direction='forward'
).pipe(lambda d: d[['F']].join(d.drop('F', 1)))
F 118.5 119.0 119.5 120.0 120.5
Expiry
2018-01-12 123.000000 0.050842 0.050842 0.050842 0.050842 0.050842
2018-01-19 123.000000 0.039526 0.039526 0.039526 0.039526 0.039526
2018-01-26 123.000000 0.039196 0.039196 0.039196 0.039196 0.039196
2018-02-02 123.000000 0.039991 0.039991 0.039991 0.039991 0.039991
2018-02-23 123.000000 0.040005 0.040005 0.040005 0.040005 0.040005
2018-03-23 122.609375 0.041025 0.041000 0.040872 0.040623 0.040398