I'm trying to get all the data from image file (jpg/jpeg/gif/png/bmp etc.) use Lua's io.read() function, but I'm not having much luck as it only seems to read a small piece of the data.
As a side note all plain text files are being read just fine, so I'm assuming that the problem is with character encoding or some such thing.
Example:
local data
local fileHandle
fileHandle = io.open ( 'pic.jpg')
data = fileHandle:read('*a')
print(data)
If you're on Windows, open file as binary: io.open('pic.jpg', 'rb').
Also it is a good idea to wrap io.open() in assert() to catch errors (or do handle them otherwise, of course).
Related
In a C++/winrt project I have a large number of small svg resources to be loaded from file. Since it would be slow to reload them all from disk at each CreateResources event from the CanvasVirtualControl I have loaded them in advance and stored the data for each in an array. When CreateResources happens my intent is to load a CanvasSvgDocument for each of these by using the CanvasSvgDocument method LoadFromXml(System.string). However, If I create an svgDocument using the resourcecreator, I get an invalid argument crash when calling LoadFromXml(). The resourceCreator argument looks right (VS preview 6 now allows me to see local variables!) and the xml data string argument looks like the valid svg data, so my best guess about the crash is that the data string is the wrong format. The file data is UTF-8. If I convert that to a std::wstring as I must for the LoadFromXml argument can it still be understood as byte data?
For example, I create the std::wstring this way, given a pointer to unsigned char file data and its length in bytes:
m_data_string = std::wstring(data, data + dataLength);
When CreateResources is triggered that datastring is referenced this way:
m_svg = CanvasSvgDocument(resourceCreator);
m_svg.LoadFromXml(resourceCreator, m_data_string);
But LoadFromXml crashes with that invalid parameter error. I see that the length of the data string is correct, but of course that is the number of characters, not the actual size of the data. Could there be a conflict between the UTF-8 attribute in the svg and the fact that it is now recorded as 16-bit characters? If so, how would one load an xml document from such data?
[Update] with the suggestion that I use winrt::to_hstring. I read the unsigned char data into a std::string,
std::string cstring = std::string("");
cstring.assign(data, data + dataLength);
Then I convert that:
m_data_string = winrt::to_hstring(cstring);
And finally try to load an svg as before:
m_svg.LoadFromXml(resourceCreator, m_data_string);
And it crashes as before. I notice that in the debugger that converted string in neither case appeared to be gibberish - in both cases it read in the debugger as the expected svg data. But if this hstring is wide chars wouldn't that be a conflict with the attribute in the svg that identifies it as UTF-8?
[Update] I'm starting to wonder if anyone has ever used CanvasSvgDocument.Draw() to draw an svg loaded from a file. The files are now loading without crashing without any change to their internal encoding reference. But - they won't draw. These files - 239 of them - are UTF-8, svg 1.1, and they display nicely if opened in Edge or any browser. But if I load the file data to an hstring, create a CanvasSvgDocument and then use CanvasSvgDocument.LoadFromXml to load them, they do not draw when called by CanvasSvgDocument's draw method. Other drawing of shapes, etc. works fine during the drawing session. Here is what could be a hint: If I call GetXML() on one of these svgs after it is loaded, what is returned is just this:
<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink"></svg>
That is, the drawing information is not there. Or is this the full extent of what GetXml() is meant to return? That wouldn't seem useful. So perhaps CanvasSvgDocument.LoadFromXml(ResourceCreator, String) doesn't actually work yet?
So I'm back to asking again: is there a way to load a functional CanvasSvgDocument from file data?
My first answer here was wrong: the fault in my code above is that LoadFromXml() is a static method, and someone pointed out to me elsewhere, I was discarding the returned result. It should be theSvg = CanvasSvgDocument::LoadFromXml(string).
Having corrected that, I'm back to the problem of loading UTF-8 data in a method whose argument is a wide-character string. Changing the internal reference to UTF-16 doesn't help after all. Loading the svg with CanvasSvgDocument::LoadAsync(filestream) works, but if I want to load these without re-accessing the disk I will need to find a way to make RandomAccessFileStream from a buffer of bytes and then use LoadAsync. I think. Unless there is some other way to make LoadFromXmL() work - at present it fails with an invalid argument error.
I am trying to log excretions in a text file in JSON format. Whole file is like a JSON object (an array of customeModle Class).
It works fine for first time but for next time when I go to log into the file I have to read it then add the new object into the array then delete previous and save it again and obviously it is not a good way to log errors.
Problems
Suppose there are many errors are getting logged at a single point of time and all are reading and appending the array then writing it back to the log file then many error won't be logged for sure.
It is consuming and wasting to much cpu and ram energy.
Please suggest a way to append new objects in the existing file without overwriting it.
Many thanks for your help you may offer.
Per Apple Documentation, you can open a file (output stream) in append mode.
Given you hold a reference to file output stream outStream, you can use below method to append data:
[NSJsonSerialization writeJSONObject:myNewObject toStream:outStream options:1 error:&error]
However, I would personally use the option you are already doing - read the data in mutable object, modify and then use NSJSONSerialization to convert it back to data again. Finally, save that data to disk - replacing the original. As this keeps JSON structure intact.
I'm trying to use Twisted in a web-app, and I'm coming across an interesting issue. I'm very new to Twisted, so I'm not sure if I'm seeing a bug in Twisted, or if I just am not using it correctly.
Theoretically from the example, a File resource object can be use to both serve files from a directory, as well as provide the directory listing. So assuming I have the variables (port, reportsDir) defined elsewhere before the code snippet, I do the following:
rootResource = Resource()
rootResource.putChild("reports", File(reportsDir))
reactor.listenTCP(port, Site(rootResource))
reactor.run(installSignalHandlers=False)
Now, when I access '/reports' on my host I get a message "Request did not return bytes" in my browser with a bunch of stuff that was obviously produced by twisted, but also contains a print of a u'.....' string literal, which in fact has the directory listing in it. So the DirectoryLister is obviously creating the listing HTML, but it isn't seeing as valid by something in Twisted. It doesn't seem to like the unicode string; which was in fact produced by Twisted itself.
Do I need to set some other configuration item to get it to convert the unicode string to the necessary bytes object (or whatever), or some other approach?
Many thanks,
-D
Well, it seems like the issue is that Python will promote any string to unicode if any source string on a format was unicode. In my case, "reportsDir" was unicode because it came from a XML file, and that set it down the error path.
Changing the above line:
rootResource.putChild("reports", File(reportsDir))
to:
rootResource.putChild("reports", File(reportsDir.encode('ascii', 'ignore')))
fixed the issue. I would however suggest that the Twisted developers do a check for unicode in the constructor for File, or in the DirectoryLister simply check for unicode, and if it is then return the ascii-encoded version.
I am making an app that lets the user draw on the screen in different colors and brush sizes. I am storing the info about each drawn path in a JSON file once it has been drawn to keep it out of memory. Right now I have it parsing all existing paths, then adding the new one in and writing it all back out again. I want it to simply append the new data into the JSON file without having to read it in and parse it first, that will make it so only one path is ever in memory at a time.
I am using SBJSON, the JSONWriter has a few append functions but I think you need to have the JSON string to append it to first, not the file, meaning I would have to read in the file anyway. Is there a way to do this without reading in the file at all? I know exactly how the data is structured.
It's possible, but you have to cheat a little. You can just create a stand-alone JSON document per path, and append that to the file. So you'll have something like this in your file:
{"name":"path1", "from": [0,3], "to":[3, 9]}
{"name":"path2", "from": [0,3], "to":[3, 9]}
{"name":"path3", "from": [0,3], "to":[3, 9]}
Note that this is not ONE JSON document but THREE. Handily, however, SBJsonStreamParser supports reading multiple JSON documents in one go. Set the supportMultipleDocuments property and plug it into a SBJsonStreamParserAdapter, and off you go. This also has the benefit that if you have many, many paths in your file as you can start drawing before you're finished reading the whole file. (Because you get a callback for each path.)
You can see some information on the use case here.
I'm pretty sure its not possible...what I ended up doing was reading in the JSON file as a string then instead of wasting memory changing all that into Dictionaries and Arrays, I just looked for an instance of part of the string (ex: i wanted to insert something before the string "], "texts"" showed up) where I wanted to insert data and inserted it there and wrote it back out to file.
As far as I can tell this is the best solution.
It seems like it should be simple but as of yet I havent found a way to save the value stored in an SSIS string variable to a text file. I've looked at using the flat file destination inside of a data flow but that requires a data flow source.
Any ideas on how to do this?
Use a script task.
I just tried this. I created a File connection manager, with the connection string pointing to the file I wanted to write to. I then created a string variable containing the text to write.
I added a Script Task, specified my string variable in the Read Only Variables list, then clicked Edit Script. The script was as follows:
public void Main()
{
ConnectionManager cm = Dts.Connections["File.tmp"];
var path = cm.ConnectionString;
var textToWrite = (string)Dts.Variables["User::StringVariable"].Value;
System.IO.File.WriteAllText(path, textToWrite);
Dts.TaskResult = (int)ScriptResults.Success;
}
This worked with no problems.
Here's a little sample of some code that worked in a SQL CLR in C#. You'll need to use VB if you're on 2005 I believe. The script task also needs the read variable property set to MyVariable to make the value of your variable available to it.
// create a writer and open the file
TextWriter tw = new StreamWriter("\\\\server\\share$\\myfile.txt");
// write a line of text to the file
tw.WriteLine(Dts.Variables["MyVariable"].Value);
// close the stream
tw.Close();
All it takes is one line of code in a simple Script task. No other dependencies, such as a connection manager, are needed.
Here's what it would look like in C#:
public void Main()
{
string variableValue = Dts.Variables["TheVariable"].Value.ToString();
string outputFile = Dts.Variables["Path"].Value.ToString();
System.IO.File.WriteAllText(outputFile, variableValue);
Dts.TaskResult = (int)ScriptResults.Success;
}
Obviously the most important line here is the one containing the WriteAllText function call.
The Path variable should contain a full path + filename for the output file.
Ok, I have an answer that doesn't involve use of script task. Pick some oledb sql source you have that's simple and you have a lot of control over. Make a query that returns only one row. Then put this query in a string variable:
"select vara, ' var =: " + #[User:varIWantToSee] + "' as myvar from tablea where vara = 1"
Then in OLEDB source pick "SQL command from a variable"
For varIWantToSee make sure you initialize it with a lot characters or ssis makes a very small length for that column that it doesn't let you override. At run time varIWantToSee will get set and you can see it. Pump this all into a flat file destination and you are in business. Why do some people have to do this? Because some people need to know the value of the variables in the runtime environment, their laptop development doesn't show the variable values they need. In my case I was running this on an Azure environment that had the database accesses I needed to test. If I were microsoft I would create a task that shows the runtime variable value at that stage of the job by writing it to the ssis log file created when the package runs. If someone knows how to do that, please enlighten us.
It's possible to use a Derived Column transformation to write the value of a variable into a column. The problem is that it needs a source to drive it, and there's no stock data source you can use that just spits out a null row onto the pipeline.
So, either you repurpose a single-row source to drive the derived column transformation, or you do what another answer suggests, and do it with a Script source.
I did it the way you described. I already had a oledb connection manager defined so I used an OLE DB Source and used the SQL Command data access mode. I used a simple query:
select getdate() as dt
...just to get it out of the way. Now I know the date of my variable pull. Then I used a Derived Column Transform to make my package variables available and wrote it out to a flat file.
Elegant? No, but it gets the job done.
Lets say you don't want to mess with Script tasks and you don't have a database you can connect to just to issue a data source command like:
SELECT 'Some arbitrary text'
There are still several ways to use a Process task for something as simple as writing a line of text to a file. For example you can use PowerShell with an input variable built using the following expression:
"'"+REPLACE(#[User::Text],"'","''")+"' > '"+REPLACE(#[User::Filename],"'","''")+"'"
Notice I escaped the filename because single quotes are legal there. Also note I used '>' for redirecting which overwrites the file if it exists. If I wanted to append I'd use '>>'.
Initially I had trouble with this method when User::Text contained multiple lines. It turns out you need some extra EOL characters after your filename when a command spans lines. Like this:
"'"+REPLACE(#[User::Text],"'","''")+"' > '"+REPLACE(#[User::Filename],"'","''")+"'\r\n\r\n"
Using cmd.exe with echo is a bit more precarious but can also work in certain circumstances and has much less overhead.
P.S. I've noticed with some versions of PowerShell that StandardInputVariable content is ignored without this:
-Command -
in the Arguments box. A lone minus sign as a Command argument is 'magic' and documented at https://learn.microsoft.com/en-us/powershell/scripting/powershell.exe-command-line-help. I believe all versions of PowerShell accept this param so even if it's not required for your version you may want to include it since it shouldn't break anything and may keep your code from breaking if PowerShell is updated to a version that requires it.