Why when I upload file with apollo-server the file is uploaded but the file is 0kb? - file-upload

I tried to solve the problem but I don't understand why the file is uploaded but his size is 0Kb.
I see this code in the tutorial but he works on that tutorial but, is not worked for me
const { ApolloServer, gql } = require('apollo-server');
const path = require('path');
const fs = require('fs');
const typeDefs = gql`
type File {
url: String!
}
type Query {
hello: String!
}
type Mutation {
fileUpload(file: Upload!): File!
}
`;
const resolvers = {
Query: {
hello: () => 'Hello world!',
},
Mutation: {
fileUpload: async (_, { file }) => {
const { createReadStream, filename, mimetype, encoding } = await file;
const stream = createReadStream();
const pathName = path.join(__dirname, `/public/images/${filename}`);
await stream.pipe(fs.createWriteStream(pathName));
return {
url: `http://localhost:4000/images/${filename}`,
};
},
},
};
const server = new ApolloServer({
typeDefs,
resolvers,
});
server.listen().then(({ url }) => {
console.log(`🚀 Server ready at ${url}`);
});
then when I upload the file, it is uploaded, but the file is 0kb
like this

What is happening is the resolver is returning before the file has uploaded, causing the server to respond before the client has finished uploading. You need to promisify and await the file upload stream events in the resolver.
Here is an example:
https://github.com/jaydenseric/apollo-upload-examples/blob/c456f86b58ead10ea45137628f0a98951f63e239/api/server.js#L40-L41
In your case:
const resolvers = {
Query: {
hello: () => "Hello world!",
},
Mutation: {
fileUpload: async (_, { file }) => {
const { createReadStream, filename } = await file;
const stream = createReadStream();
const path = path.join(__dirname, `/public/images/${filename}`);
// Store the file in the filesystem.
await new Promise((resolve, reject) => {
// Create a stream to which the upload will be written.
const writeStream = createWriteStream(path);
// When the upload is fully written, resolve the promise.
writeStream.on("finish", resolve);
// If there's an error writing the file, remove the partially written
// file and reject the promise.
writeStream.on("error", (error) => {
unlink(path, () => {
reject(error);
});
});
// In Node.js <= v13, errors are not automatically propagated between
// piped streams. If there is an error receiving the upload, destroy the
// write stream with the corresponding error.
stream.on("error", (error) => writeStream.destroy(error));
// Pipe the upload into the write stream.
stream.pipe(writeStream);
});
return {
url: `http://localhost:4000/images/${filename}`,
};
},
},
};
Note that it’s probably not a good idea to use the filename like that to store the uploaded files, as future uploads with the same filename will overwrite earlier ones. I'm not really sure what will happen if two files with the same name are uploaded at the same time by two clients.

Related

S3 to IPFS from Pinata

I am trying to upload a lot of files from S3 to IPFS via Pinata. I haven't found in Pinata documentation something like that.
This is my solution, using the form-data library. I haven't tested it yet (I will do it soon, I need to code some things).
Is it a correct approach? anyone who has done something similar?
async uploadImagesFolder(
items: ItemDocument[],
bucket?: string,
path?: string,
) {
try {
const form = new FormData();
for (const item of items) {
const file = getObjectStream(item.tokenURI, bucket, path);
form.append('file', file, {
filename: item.tokenURI,
});
}
console.log(`Uploading files to IPFS`);
const pinataOptions: PinataOptions = {
cidVersion: 1,
};
const result = await pinata.pinFileToIPFS(form, {
pinataOptions,
});
console.log(`Piñata Response:`, JSON.stringify(result, null, 2));
return result.IpfsHash;
} catch (e) {
console.error(e);
}
}
I had the same problem
So, I have found this: https://medium.com/pinata/stream-files-from-aws-s3-to-ipfs-a0e23ffb7ae5
But in the article If am not wrong, is used a different version to the JavaScript AWS SDK v3 (nowadays the most recent: https://docs.aws.amazon.com/AWSJavaScriptSDK/v3/latest/index.html).
This is for the Client side with TypeScript:
If you have this version, for me works this code snippet:
export const getStreamObjectInAwsS3 = async (data: YourParamsType) => {
try {
const BUCKET = data.bucketTarget
const KEY = data.key
const client = new S3Client({
region: 'your-region',
credentials: {
accessKeyId: 'your-access-key',
secretAccessKey: 'secret-key'
}
})
const resource = await client.send(new GetObjectCommand({
Bucket: BUCKET,
Key: KEY
}))
const response = resource.Body
if (response) {
return new Response(await response.transformToByteArray()).blob()
}
return null
} catch (error) {
return null
}
}
With the previous code, you can get the Blob Object for pass it to the File object with this method and get the URL resource using the API:
export const uploadFileToIPFS = async(file: Response) => {
const url = `https://api.pinata.cloud/pinning/pinFileToIPFS`
const data = new FormData()
data.append('file', file)
try {
const response = await axios.post(url, data, {
maxBodyLength: Infinity,
headers: {
pinata_api_key: 'your-api',
pinata_secret_api_key: 'your-secret'
},
data: data
})
return {
success: true,
pinataURL: `https://gateway.pinata.cloud/ipfs/${ response.data.IpfsHash }`
}
} catch (error) {
console.log(error)
return null
}
}
I have found this solution from this nice article and you can explore other implementations (including the Node.js side)

React native content uri to base64 string

I'm trying to upload files using RN Document Picker.
Once I get those files selected, I need to turn them to base64 string so I can send it to my API.
const handlePickFiles = async () => {
if (await requestExternalStoreageRead()) {
const results = await DocumentPicker.pickMultiple({
type: [
DocumentPicker.types.images,
DocumentPicker.types.pdf,
DocumentPicker.types.docx,
DocumentPicker.types.zip,
],
});
const newUploadedFile: IUploadedFile[] = [];
for (const res of results) {
console.log(JSON.stringify(res, null, 2));
newUploadedFile.push({
name: res.name,
type: res.type as string,
size: res.size as number,
extension: res.type!.split('/')[1],
blob: res.uri, <<-- Must turn this in base64 string
});
}
setUploadedFiles(newUploadedFile);
console.log(newUploadedFile);
}
}
};
The document picker returns content uri (content://...)
They lists this as an example of handling blob data and base64:
let data = new FormData()
data.append('image', {uri: 'content://path/to/content', type: 'image/png', name: 'name'})
const response = await fetch(url, {
method: 'POST',
headers: {
'Content-Type': 'multipart/form-data',
},
body: data
})
Where they basically say that you don't need to use blob or base64 when using multipart/form-data as content type. However, my graphql endpoint cannot handle multipart data and I don't have time to rewrite the whole API. All I want is to turn it to blob and base64 string, even if other ways are more performant.
Searching for other libraries, all of them are no longer maintained, or has issues with new versions of android. RN Blob Utils is the latest npm that was no longer maintained.
I tried to use RN Blob Utils but I either get errors, wrong data type, or the file uploads but is corrupted.
Some other things I found is that I can use
fetch(res.uri).then(response => {response.blob()})
const response = await ReactNativeBlobUtil.fetch('GET', res.uri);
const data = response.base64();
ReactNativeBlobUtil.wrap(decodeURIComponent(blob))
///----
const blob = ReactNativeBlobUtil.fs.readFile(res.uri, 'base64');
But I can't do anything with that blob file.
What is the simplest way to uplaod files from document picker as base64 format? Is it possible to avoid using external storage permission?
You don't need to the third-party package to fetch BLOB data
const blob = await new Promise((resolve, reject) => {
const xhr = new XMLHttpRequest();
xhr.onload = function () {
resolve(xhr.response);
};
xhr.onerror = function (e) {
reject(new TypeError("Network request failed"));
};
xhr.responseType = "blob";
xhr.open("GET", "[LOCAL_FILE_PATH]", true);
xhr.send(null);
});
// Code to submit blob file to server
// We're done with the blob, close and release it
blob.close();
I ended up using react-native-blob-util
const res = await DocumentPicker.pickSingle({
type: [
DocumentPicker.types.images,
DocumentPicker.types.pdf,
DocumentPicker.types.docx,
DocumentPicker.types.zip,
],
});
const newUploadedFile: IUploadedFile[] = [];
const fileType = res.type;
if (fileType) {
const fileExtension = fileType.substr(fileType.indexOf('/') + 1);
const realURI = Platform.select({
android: res.uri,
ios: decodeURI(res.uri),
});
if (realURI) {
const b64 = await ReactNativeBlobUtil.fs.readFile(
realURI,
'base64',
);
const filename = res.name.replace(/\s/g, '');
const path = uuid.v4();
newUploadedFile.push({
name: filename,
type: fileType,
size: res.size as number,
extension: fileExtension,
blob: b64,
path: Array.isArray(path) ? path.join() : path,
});
} else {
throw new Error('Failed to process file');
}
} else {
throw new Error('Failed to process file');
}

How to send `application/zip` in react-native

I want to generate a zip file and then POST it up. I am trying to leverage this lib JSZip to create a zip with several files in it. Here is what I have right now.
handlePendingItems = async (routeId: number) => {
try {
await dbClient.addLogLine(`Need to delete photos.`);
const [debugLogs, pendingTasks, pendingUploads] = await this.getPendingItems();
const dbVersion = await dbClient.getUserSchemaVersion();
let appInfo = `Database Version = ${dbVersion}\n`;
appInfo += `App Version = ${version}`;
const zip = new JSZip();
zip.file('app-info.txt', appInfo);
if (debugLogs.length) {
const debugLogsCsv = convertDataToCSV(debugLogs);
zip.file('debug-logs.csv', debugLogsCsv);
}
if (pendingTasks.length) {
const pendingTasksCsv = convertDataToCSV(pendingTasks);
zip.file('pending-tasks.csv', pendingTasksCsv);
}
if (pendingUploads.length) {
const pendingUploadsCsv = convertDataToCSV(pendingUploads);
zip.file('pending-uploads.csv', pendingUploadsCsv);
}
const test = await zip.generateAsync({ type: 'binarystring' });
console.log(test);
const res = PhotoService.uploadLogs(routeId, test);
// const zipFile = await zip.generateAsync({
// type: 'blob',
// compression: 'DEFLATE',
// compressionOptions: {
// level: 9
// }
// });
} catch (error) {
console.log(error);
}
};
I can't use the type: 'blob' in react-native, it throws an error as unsupported on this platform. That being said I am trying to use other types to pass to react-native-fetch-blob. To be posted via the following code.
uploadLogs(driverRouteId: number, logs: string) {
const store = getStore();
const token = store.getState()?.auth?.token || '';
return new Promise((resolve, reject) => {
RNFetchBlob.fetch(
'POST',
`${API_HOST}driver/route/${driverRouteId}/logs`,
{
'Content-Type': 'application/zip',
Authorization: `Bearer ${token}`
},
logs
).then(
(res) => {
console.log(res);
resolve(res);
},
(err) => {
console.log(err);
debugger;
reject(err);
}
);
});
}
I read that fetch blob will attempt to parse base64 string if the content-encoding is a specific type which in this case has to be application/zip (so no parsing in this case) so I am trying to call generateAsync with type string however, the zip the file the server stores is in an unsupported format. How can I generate the zip file containing the three csv files and the txt file and send it using either fetch blob or just plain axios in react-native.
Figured out a solution using a different library to zip.
npm install react-native-zip-archive --save
npm install react-native-fs --save
npm install rn-fetch-blob --save
import RNFetchBlob from 'rn-fetch-blob';
import { zip } from 'react-native-zip-archive';
import RNFS from 'react-native-fs';
const pathsToZip = [];
const zipPath = `${RNFS.DocumentDirectoryPath}/test.zip`;
const appInfo = 'App info string to put in txt file'
const appInfoPath = RNFS.DocumentDirectoryPath + '/appInfo.txt';
await RNFS.writeFile(appInfoPath, appInfo, 'utf8');
pathsToZip.push(appInfoPath);
await zip(pathsToZip, zipPath);
await uploadLogs(zipPath) {
return new Promise((resolve, reject) => {
RNFetchBlob.fetch(
'POST',
`postpath`,
{
'Content-Type': 'application/zip',
},
RNFetchBlob.wrap(zipPath)
).then(
(res) => {
console.log(res);
resolve(res);
},
(err) => {
console.log(err);
debugger;
reject(err);
}
);
});
}

React Native: Failed to execute 'append' on 'FormData': parameter 2 is not of type 'Blob'. at new ApolloError

I am trying to upload image from my react native app to graphql by using Apollo client with createUploadLink(). When I am trying to mutate data by passing a ReactNativeFile as a variable, then it says
"network request failed: Failed to execute 'append' on 'FormData': parameter 2 is not of type 'Blob'. at new ApolloError ".
This this the mutation which i am trying to use
mutation publishPost(
$content: String!
$LocationInput: LocationInput!
$InputPostAttachment: [InputPostAttachment!]
) {
publishPost(
content: $content
location: $LocationInput
attachments: $InputPostAttachment
) {
content
}
}
InputPostAttachment has type
type InputPostAttachment {
type: PostAttachmentType!
file: Upload!
}
Apollo client settings and i am using apollo-upload-client
const httpLink = createUploadLink({
uri: 'http://localhost:8000/graphql',
});
const authLink = setContext(async (headers: any) => {
const token = await getToken();
return {
...headers,
headers: {
authorization: token ? `Bearer ${token}` : null,
},
};
});
const link = authLink.concat(httpLink);
// create an inmemory cache instance for caching graphql data
const cache = new InMemoryCache();
// instantiate apollo client with apollo link instance and cache instance
export const client = new ApolloClient({
link,
cache,
});
File upload Function and i am using react-native-image-crop-picker for multi image selection
const [image, setimage] = useState([]);
const _pickImage = () => {
ImagePicker.openPicker({
includeBase64: true,
multiple: true,
}).then((images: any) => {
let imageData: any = [];
images.map((data: any) => {
const file = new ReactNativeFile({
uri: data.path,
name: data.filename,
type: data.mime,
});
imageData.push({
type: 'IMAGE',
file: file,
});
});
setimage(imageData);
console.log(images);
});
};
const handlePost = async () => {
const InputPostAttachment: any = [...image];
const LocationInput = {
place: place,
vicinity: vicinity,
province: province,
};
publishPost({variables: {content, LocationInput, InputPostAttachment}})
.then(({data}) => {
console.log(data);
props.navigation.navigate('Home');
})
.catch((err) => {
console.log('err happened');
console.log(err);
});
};
could someone please help me out from this?
In addition to the chrome debugger issue, this error also happens on the expo web.
To anyone uploading images on expo web (or react-native web), here's a working solution:
/** Load image from camera/roll. */
const result = await ImagePicker.launchImageLibraryAsync({
mediaTypes: ImagePicker.MediaTypeOptions.All,
allowsEditing: true,
quality: 1,
});
if (result.cancelled) {
return;
}
/** web platform: blob. */
const convertBase64ToBlob = async (base64) => {
const response = await fetch(base64);
const blob = await response.blob();
return blob;
};
/** android/ios platform: ReactNativeFile.*/
const createReactNativeFile = (uri) => {
const file = new ReactNativeFile({
uri,
type: mime.lookup(uri) || 'image',
name: `file-${Date.now()}`,
});
return file;
};
/** Use blob for web, ReactNativeFile otherwise. */
const file = Platform.OS === 'web'
? await convertBase64ToBlob(result.uri)
: createReactNativeFile(result.uri);
/** Upload image with apollo. */
mutate({ variables: { file } });
On the web platform, ImagePicker returns a base64 value instead of a file path. This problem doesn't happen if the platform is Android or iOS, as ImagePicker returns a file path, which is expected by apollo-upload-client.
The solution is to detect if the URI is base64 (which happens when the platform is "web") and convert it to a blob.
My apollo-client was configured using apollo-boost and i was using chrome debugger to intercept the network was causing me this issue.
To be more specific I was using the below code to get the network requests sent by my app in the chrome debugger
global.XMLHttpRequest =
global.originalXMLHttpRequest || global.XMLHttpRequest;
global.FormData = global.originalFormData || global.FormData;
if (window.FETCH_SUPPORT) {
window.FETCH_SUPPORT.blob = false;
} else {
global.Blob = global.originalBlob || global.Blob;
global.FileReader = global.originalFileReader || global.FileReader;
}
apollo-upload-client wont send the data in multipart data if we are using chrome debugger. We will face network issue.This issue has the answer. or I had not removed apollo-boost and some part of my app was using it that was also a issue.

nativescript image upload to s3 encoding issue

I'm having issues uploading an image from Nativescript to AWS, and I'm pretty sure it's a configuration issue.
select an image
const context = imagepicker.create({
mode: 'single' // use "multiple" for multiple selection
});
await context.authorize();
const selection: Array<ImageAsset> = await context.present();
const imageAsset = selection[0];
const source: ImageSource = await new ImageSource().fromAsset(imageAsset);
const fileLocation = imageAsset.android ? imageAsset.android : imageAsset.ios;
const fileType = mime.extension(mime.lookup(fileLocation));
const image = source.toBase64String(fileType);
console.log(image);
image at this point is: iVBORw0KGgoAAAANSUhEUgAAB4AAAASwCAIAAACVUsChAAAAA3NCSVQI...
image location at this point is: /storage/emulated/0/DCIM/Screenshots/Screenshot_20181106-150854.png
const fileLocation = imageAsset.android ? imageAsset.android : imageAsset.ios;
const signedUrl = await this.getSignedUrl(fileLocation);
Backend Code to get signedURL
const getSignedUrlPromise = (operation, params) => {
return new Promise((resolve, reject) => {
s3.getSignedUrl(operation, params, (err, url) => {
err ? reject(err) : resolve(url);
});
});
}
const params = {
Bucket: BUCKET_NAME,
Key: `abc123/456/3/${fileName}`,
ContentType: contentType,
ContentEncoding: 'base64'
}
const url = await getSignedUrlPromise('putObject', params).catch(err => {
console.log('error', JSON.stringify(err))
return {
statusCode: 400,
body: JSON.stringify(err)
}
});
console.log('success', url);
return {
statusCode: 200,
body: JSON.stringify({ url: url })
}
signedUrl at this point is:
https://myproject.s3.amazonaws.com/abc123/456/3/Screenshot_20181106-150854.png?AWSAccessKeyId=xxxxxxxxxxxxxx&Content-Encoding=base64&Content-Type=image%2Fpng&Expires=1555517358&Signature=yyyyyyyy&x-amz-security-token=long_token
Then, using the signedURL, i upload the image:
const mimeType = mime.lookup(fileLocation);
this.http.put(signedUrl, image, {
headers: {
'Content-Type': mimeType,
'Content-Encoding': 'base64'
}
}).subscribe((resp) => {
console.log('resp2', resp);
});
}
When I open the file, this is what I see
and the meta-data on the S3 object looks correct
When I download the file and open it in NP++, I see the base64 value.
iVBORw0KGgoAAAANSUhEUgAAB4AAAASwCAIAAACVUsChAAAAA3NCSVQI...
I also cannot open the downloaded image
ATTEMPT 2
I saw where some people were using buffers, so I changed my image code to
const image = Buffer.from(source.toBase64String(fileType).replace(/^data:image\/\w+;base64,/, ''), 'base64');
which the image is still broken, and when I download and open the file using NP++ I see
{"type":"Buffer","data":[137,80,78,71,13,10,26,10,0,0,0,13,73,72,68,82,0,0,7,128,0,0,4,176,8,2,0,0,0