AWS SDK 3 WriteGetObjectResponseCommand issue - amazon-s3

On my current project we are trying to migrate to AWS SDK version 3 (Javascript). We are using s3 object lambdas and have faced and issue with WriteGetObjectResponseCommand command.
WriteGetObjectResponseCommand code:
import { S3Client, S3 } from '#aws-sdk/client-s3';
const s3Client = new S3Client({});
async function passTransformedPhoto(
body: Buffer,
outputRoute: string,
outputToken: string,
): Promise<WriteGetObjectResponseCommandOutput> {
try {
const writeGetObjectParams: WriteGetObjectResponseCommandInput = {
RequestRoute: outputRoute,
RequestToken: outputToken,
Body: body,
};
const command = new WriteGetObjectResponseCommand(writeGetObjectParams);
return s3Client.send(command);
} catch (e) {
console.log('Error happens in passTransformedPhoto', e);
}
The same issue with the aggregated client:
import { S3Client, S3 } from '#aws-sdk/client-s3';
const s3Aggregated = new S3({});
async function passTransformedPhoto(body: Buffer, outputRoute: string, outputToken: string) {
try {
const result = await s3Aggregated.writeGetObjectResponse({
RequestRoute: outputRoute,
RequestToken: outputToken,
Body: body,
});
return result;
} catch (e) {
console.log('Error happens in passTransformedPhoto', e);
}
}
Error:
INFO TypeError: Cannot destructure property 'service' of 'bucketName' as it is undefined.
at getEndpointFromArn (/var/task/node_modules/#aws-sdk/middleware-bucket-endpoint/dist/cjs/bucketHostname.js:26:13)
at Object.bucketHostname (/var/task/node_modules/#aws-sdk/middleware-bucket-endpoint/dist/cjs/bucketHostname.js:17:13)
at /var/task/node_modules/#aws-sdk/middleware-bucket-endpoint/dist/cjs/bucketEndpointMiddleware.js:47:67
at processTicksAndRejections (internal/process/task_queues.js:95:5)
Environment:
#aws-sdk/client-s3: 3.18.0
Node.js: v 14.17.0
typescript: 4.3.2
Any help or clarification on this issue is much appreciated.

Maybe try including all the expected S3Client config options:
const config = {
region: <region>,
bucketEndpoint: true,
}
const client = new S3Client(config);

Related

Nuxt 3 - Server not ready on mount, Volar engine confused

I have two issues that may or may not be related.
Overview
Folder Structure
pages
|---user.vue
server
|---api
|---profile.get.ts
|---profile.post.ts
The tech is Nuxt3 using the server and Supabase.
profile.get.ts
import { serverSupabaseClient, serverSupabaseUser } from "#supabase/server"
import { Database } from "~~/types/supabase"
export default defineEventHandler(async (event) => {
try {
const supabase = serverSupabaseClient<Database>(event)
const user = await serverSupabaseUser(event)
const query = getQuery(event)
const { data, error } = await supabase.from('profiles').select('*').eq('email', query.email).single()
if (error) throw { status: error.code, message: error.message }
return { displayName: data.display_name, avatarUrl: data.avatar_url }
} catch (err) {
console.error('Handled Error:', err)
}
})
profile.post.ts
import { serverSupabaseClient } from "#supabase/server"
import { Database } from "~~/types/supabase"
export default defineEventHandler(async (event) => {
const supabase = serverSupabaseClient<Database>(event)
const { displayName, avatarUrl, email }: { displayName: string, avatarUrl: string, email: string } = await readBody(event)
const { error } = await supabase.from('profiles').update({ display_name: displayName, avatar_url: avatarUrl }).match({ email })
if (error) throw new Error(error.message)
return { status: 200 }
})
user.vue Snippet
onMounted(() => {
setTimeout(() => {
getProfile()
}, 100) // Fails when around 50 or less
})
async function getProfile() {
const { data, error } = await useFetch('/api/profile', { method: 'GET', params: { email: user.value?.email } })
console.log(data.value)
console.log(error.value)
displayName.value = data.value!.displayName || ''
avatarUrl.value = data.value!.avatarUrl || ''
}
Problem 1
When user.vue mounts, I want to call my Nuxt API (profile.get.ts) and fetch user data (display name, avatar url) from the Supabase database. However, I receive this error when fetching on mount: FetchError: 404 Cannot find any route matching /api/profile. (/api/profile). However, if I use setTimeout to 100ms, it fetches fine. That makes me think the API server is simply not ready, but the documentation doesn't mention that and encourages fetching during lifecycle.
Problem 2
Volar seems to be confused about the typing of data from getProfile().
Property 'displayName' does not exist on type '{ status: number; } | { displayName: string | null; avatarUrl: string | null; }'.
Property 'displayName' does not exist on type '{ status: number; }'.ts(2339)
However, this is the typing from profile.post.ts even though I'm using profile.get.ts.
Current Behavior
Without setTimeout at 100ms or greater, it will fail with the 404 message
With setTimeout at 100ms or greater, or with getProfile() called from a button, there is no issue, even with the TypeScript errors, etc.
Desired Behavior
TypeScript correctly recognizes the proper endpoint (profiles.get.ts since I'm calling it with get)
Data can be fetched on mount from the API without the use of setTimeout

Problems with deploying first smart contracts

Good day everyone!
I'm trying to deploy the first smart contracts according to the helloWorld manuals, but I'm getting an error please see the snippet. Tell me, please, what can be done about this? Tried lot of way to fix it but failed.
Thanks in advance!
import { Command } from ""commander"";
import prompts, { PromptObject } from ""prompts"";
import { isNumeric, Migration } from ""./utils"";
const program = new Command();
const migration = new Migration();
async function main() {
const promptsData: PromptObject[] = [];
program
.allowUnknownOption()
.option(""-kn, --key_number <key_number>"", ""Public key number"")
.option(
""-b, --balance <balance>"",
""Initial balance in EVERs (will send from Giver)"",
);
program.parse(process.argv);
const options = program.opts();
if (!options.key_number) {
promptsData.push({
type: ""text"",
name: ""keyNumber"",
message: ""Public key number"",
validate: value => (isNumeric(value) ? true : ""Invalid number""),
});
}
if (!options.balance) {
promptsData.push({
type: ""text"",
name: ""balance"",
message: ""Initial balance (will send from Giver)"",
validate: value => (isNumeric(value) ? true : ""Invalid number""),
});
}
const response = await prompts(promptsData);
const keyNumber = +(options.key_number || response.keyNumber);
const balance = +(options.balance || response.balance);
const signer = (await locklift.keystore.getSigner(keyNumber.toString()))!;
let accountsFactory = locklift.factory.getAccountsFactory(""Account"");
const { account: Account } = await accountsFactory.deployNewAccount({
publicKey: signer.publicKey,
initParams: {
_randomNonce: locklift.utils.getRandomNonce(),
},
constructorParams: {},
value: locklift.utils.toNano(balance),
});
migration.store(Account, ""account"");
console.log(`Account deployed at: ${Account.address}`);
}
main()
.then(() => process.exit(0))
.catch(e => {
console.log(e);
process.exit(1);
});"
Crashes on an attempt to collect Account contract assembly artefacts
There is no such thing in the build folder!

Duplicate identifier 'alpha'.deno-ts(2300) Unexpected keyword or identifier

I am trying to use Alpha Vantage NPM package inside my Deno application. I tried to use SkyPack version of it. But it gives me the following error:
Duplicate identifier 'alpha'.deno-ts(2300)
Unexpected keyword or identifier.
This is the code I am using:
import alphavantageTs from 'https://cdn.skypack.dev/alphavantage-ts';
export class StockTimeSeries{
alpha = new alphavantageTs ("ASLDVIWXGEWFWNZG");
alpha.stocks.intraday("msft").then((data: any) => {
console.log(data);
});
alpha.stocks.batch(["msft", "aapl"]).then((data: any) => {
console.log(data);
});
alpha.forex.rate("btc", "usd").then((data: any) => {
console.log(data);
});
alpha.crypto.intraday("btc", "usd").then((data: any) => {
console.log(data);
});
alpha.technicals.sma("msft", "daily", 60, "close").then((data: any) => {
console.log(data);
});
alpha.sectors.performance().then((data: any) => {
console.log(data);
});
}
It looks like SkyPack is responding with a 401 for one of the sub-dependencies for that module. I'm also not even sure it's compatible with Deno.
That said, here's the repo source for the module, and here's the documentation for the API. It looks like it's just a simple REST API which discriminates requests by query parameters, so you can make your own Deno client without too much effort using that module as a template. I'll give you some starter code:
TS Playground
export type Params = NonNullable<ConstructorParameters<typeof URLSearchParams>[0]>;
class AlphaVantageNS { constructor (protected readonly api: AlaphaVantage) {} }
class Forex extends AlphaVantageNS {
rate (from_currency: string, to_currency: string) {
return this.api.query({
function: 'CURRENCY_EXCHANGE_RATE',
from_currency,
to_currency,
});
}
}
export class AlaphaVantage {
#token: string;
constructor (token: string) {
this.#token = token;
}
async query <Result = any>(params: Params): Promise<Result> {
const url = new URL('https://www.alphavantage.co/query');
const usp = new URLSearchParams(params);
usp.set('apikey', this.#token);
url.search = usp.toString();
const request = new Request(url.href);
const response = await fetch(request);
if (!response.ok) throw new Error('Response not OK');
return response.json();
}
forex = new Forex(this);
}
// Use:
const YOUR_API_KEY = 'demo';
const alpha = new AlaphaVantage(YOUR_API_KEY);
alpha.forex.rate('BTC', 'USD').then(data => console.log(data));

Nestjs - file upload with fastify multipart

I am trying to upload multiple files with nestjs using the fastify adapter. I can do so following the tutorial in this link -article on upload
Now this does the job of file upload using fastify-multipart, but I couldnt make use of the request validations before uploading,
for example, here is my rule-file-models (which later I wanted to save to postgre)
import {IsUUID, Length, IsEnum, IsString, Matches, IsOptional} from "class-validator";
import { FileExtEnum } from "./enums/file-ext.enum";
import { Updatable } from "./updatable.model";
import {Expose, Type} from "class-transformer";
export class RuleFile {
#Expose()
#IsUUID("4", { always: true })
id: string;
#Expose()
#Length(2, 50, {
always: true,
each: true,
context: {
errorCode: "REQ-000",
message: `Filename shouldbe within 2 and can reach a max of 50 characters`,
},
})
fileNames: string[];
#Expose()
#IsEnum(FileExtEnum, { always: true, each: true })
fileExts: string[];
#IsOptional({each: true, message: 'File is corrupated'})
#Type(() => Buffer)
file: Buffer;
}
export class RuleFileDetail extends RuleFile implements Updatable {
#IsString()
#Matches(/[aA]{1}[\w]{6}/)
recUpdUser: string;
}
And I wanted to validate the multipart request and see if these are set properly.
I cannot make it to work with event subscription based approach. Here are a few things I tried - adding the interceptor, to check for the request
#Injectable()
export class FileUploadValidationInterceptor implements NestInterceptor {
intercept(context: ExecutionContext, next: CallHandler): Observable<any> {
const req: FastifyRequest = context.switchToHttp().getRequest();
console.log('inside interceptor', req.body);
// content type cmes with multipart/form-data;boundary----. we dont need to valdidate the boundary
// TODO: handle split errors based on semicolon
const contentType = req.headers['content-type'].split(APP_CONSTANTS.CHAR.SEMI_COLON)[0];
console.log(APP_CONSTANTS.REGEX.MULTIPART_CONTENT_TYPE.test(contentType));
const isHeaderMultipart = contentType != null?
this.headerValidation(contentType): this.throwError(contentType);
**// CANNOT check fir req.file() inside this, as it throws undefined**
return next.handle();
}
headerValidation(contentType) {
return APP_CONSTANTS.REGEX.MULTIPART_CONTENT_TYPE.test(contentType) ? true : this.throwError(contentType);
}
throwError(contentType: string) {
throw AppConfigService.getCustomError('FID-HEADERS', `Request header does not contain multipart type:
Provided incorrect type - ${contentType}`);
}
}
I wasnt able to check req.file() in the above interceptor. It throws as undefined. I tried to follow the fastify-multipart
But I wasnt able to get the request data in a prehandler as provided in the documentation for fastify-multipart
fastify.post('/', async function (req, reply) {
// process a single file
// also, consider that if you allow to upload multiple files
// you must consume all files othwise the promise will never fulfill
const data = await req.file()
data.file // stream
data.fields // other parsed parts
data.fieldname
data.filename
data.encoding
data.mimetype
// to accumulate the file in memory! Be careful!
//
// await data.toBuffer() // Buffer
//
// or
await pump(data.file, fs.createWriteStream(data.filename))
I tried getting via by registering a prehandler hook of my own like this (executed as iife)
(async function bootstrap() {
const appConfig = AppConfigService.getAppCommonConfig();
const fastifyInstance = SERVERADAPTERINSTANCE.configureFastifyServer();
// #ts-ignore
const fastifyAdapter = new FastifyAdapter(fastifyInstance);
app = await NestFactory.create<NestFastifyApplication>(
AppModule,
fastifyAdapter
).catch((err) => {
console.log("err in creating adapter", err);
process.exit(1);
});
.....
app.useGlobalPipes(
new ValidationPipe({
errorHttpStatusCode: 500,
transform: true,
validationError: {
target: true,
value: true,
},
exceptionFactory: (errors: ValidationError[]) => {
// send it to the global exception filter\
AppConfigService.validationExceptionFactory(errors);
},
}),
);
app.register(require('fastify-multipart'), {
limits: {
fieldNameSize: 100, // Max field name size in bytes
fieldSize: 1000000, // Max field value size in bytes
fields: 10, // Max number of non-file fields
fileSize: 100000000000, // For multipart forms, the max file size
files: 3, // Max number of file fields
headerPairs: 2000, // Max number of header key=>value pairs
},
});
(app.getHttpAdapter().getInstance() as FastifyInstance).addHook('onRoute', (routeOptions) => {
console.log('all urls:', routeOptions.url);
if(routeOptions.url.includes('upload')) {
// The registration actually works, but I cant use the req.file() in the prehandler
console.log('###########################');
app.getHttpAdapter().getInstance().addHook('preHandler', FilePrehandlerService.fileHandler);
}
});
SERVERADAPTERINSTANCE.configureSecurity(app);
//Connect to database
await SERVERADAPTERINSTANCE.configureDbConn(app);
app.useStaticAssets({
root: join(__dirname, "..", "public"),
prefix: "/public/",
});
app.setViewEngine({
engine: {
handlebars: require("handlebars"),
},
templates: join(__dirname, "..", "views"),
});
await app.listen(appConfig.port, appConfig.host, () => {
console.log(`Server listening on port - ${appConfig.port}`);
});
})();
Here is the prehandler,
export class FilePrehandlerService {
constructor() {}
static fileHandler = async (req, reply) => {
console.log('coming inside prehandler');
console.log('req is a multipart req',await req.file);
const data = await req.file();
console.log('data received -filename:', data.filename);
console.log('data received- fieldname:', data.fieldname);
console.log('data received- fields:', data.fields);
return;
};
}
This pattern of registring and gettin the file using preHandler works in bare fastify application. I tried it
Bare fastify server:
export class FileController {
constructor() {}
async testHandler(req: FastifyRequest, reply: FastifyReply) {
reply.send('test reading dne');
}
async fileReadHandler(req, reply: FastifyReply) {
const data = await req.file();
console.log('field val:', data.fields);
console.log('field filename:', data.filename);
console.log('field fieldname:', data.fieldname);
reply.send('done');
}
}
export const FILE_CONTROLLER_INSTANCE = new FileController();
This is my route file
const testRoute: RouteOptions<Server, IncomingMessage, ServerResponse, RouteGenericInterface, unknown> = {
method: 'GET',
url: '/test',
handler: TESTCONTROLLER_INSTANCE.testMethodRouteHandler,
};
const fileRoute: RouteOptions = {
method: 'GET',
url: '/fileTest',
preHandler: fileInterceptor,
handler: FILE_CONTROLLER_INSTANCE.testHandler,
};
const fileUploadRoute: RouteOptions = {
method: 'POST',
url: '/fileUpload',
preHandler: fileInterceptor,
handler: FILE_CONTROLLER_INSTANCE.fileReadHandler,
};
const apiRoutes = [testRoute, fileRoute, fileUploadRoute];
export default apiRoutes;
Could someone let me know the right the way to get the fieldnames , validate them befr the service being called in Nestjs
Well, I have done something like this and It works great for me. Maybe it can work for you too.
// main.ts
import multipart from "fastify-multipart";
const app = await NestFactory.create<NestFastifyApplication>(
AppModule,
new FastifyAdapter(),
);
app.register(multipart);
// upload.guard.ts
import {
Injectable,
CanActivate,
ExecutionContext,
BadRequestException,
} from "#nestjs/common";
import { FastifyRequest } from "fastify";
#Injectable()
export class UploadGuard implements CanActivate {
public async canActivate(ctx: ExecutionContext): Promise<boolean> {
const req = ctx.switchToHttp().getRequest() as FastifyRequest;
const isMultipart = req.isMultipart();
if (!isMultipart)
throw new BadRequestException("multipart/form-data expected.");
const file = await req.file();
if (!file) throw new BadRequestException("file expected");
req.incomingFile = file;
return true;
}
}
// file.decorator.ts
import { createParamDecorator, ExecutionContext } from "#nestjs/common";
import { FastifyRequest } from "fastify";
export const File = createParamDecorator(
(_data: unknown, ctx: ExecutionContext) => {
const req = ctx.switchToHttp().getRequest() as FastifyRequest;
const file = req.incomingFile;
return file
},
);
// post controller
#Post("upload")
#UseGuards(UploadGuard)
uploadFile(#File() file: Storage.MultipartFile) {
console.log(file); // logs MultipartFile from "fastify-multipart"
return "File uploaded"
}
and finally my typing file
declare global {
namespace Storage {
interface MultipartFile {
toBuffer: () => Promise<Buffer>;
file: NodeJS.ReadableStream;
filepath: string;
fieldname: string;
filename: string;
encoding: string;
mimetype: string;
fields: import("fastify-multipart").MultipartFields;
}
}
}
declare module "fastify" {
interface FastifyRequest {
incomingFile: Storage.MultipartFile;
}
}
So I found a simpler alternative. I started using fastify-multer. I used it along with this awesome lib - which made me use the multer for fastify - #webundsoehne/nest-fastify-file-upload
These are the changes I made. I registered the multer content process.
app.register(multer( {dest:path.join(process.cwd()+'/upload'),
limits:{
fields: 5, //Number of non-file fields allowed
files: 1,
fileSize: 2097152,// 2 MB,
}}).contentParser);
Then in the controller - I use it as the nestjs doc says . This actually makes fasitfy work with multer
#UseInterceptors(FileUploadValidationInterceptor, FileInterceptor('file'))
#Post('/multerSample')
async multerUploadFiles(#UploadedFile() file, #Body() ruleFileCreate: RuleFileCreate) {
console.log('data sent', ruleFileCreate);
console.log(file);
// getting the original name of the file - no matter what
ruleFileCreate.originalName = file.originalname;
return await this.fileService.fileUpload(file.buffer, ruleFileCreate);
}
BONUS - storing the file in local and storing it in DB - Please refer
github link

How to parse serverless.yml file in script

I need to read the serverless.yml config for use in some test mocks.
The following worked until a recent change:
const serverless = new Serverless()
await serverless.init()
const service = await serverless.variables.populateService()
How does one read the file now? There is an astounding lack of documentation in regards to using serverless progamically.
Well I ended up taking some code from the AppSync emulator package. I am not sure it covers it does a full parsing but it does the job for me.
import Serverless from 'serverless'
import path from 'path'
import fs from 'fs'
class ConfigServerless extends Serverless {
async getConfig(servicePath) {
this.processedInput = {
commands: [],
options: { stage: 'dev' }
}
this.config.servicePath = servicePath
this.pluginManager.setCliOptions(this.processedInput.options)
this.pluginManager.setCliCommands(this.processedInput.commands)
await this.service.load(this.processedInput)
this.pluginManager.validateCommand(this.processedInput.commands)
return this.variables
.populateService(this.pluginManager.cliOptions)
.then(() => {
this.service.mergeArrays()
this.service.setFunctionNames(this.processedInput.options)
this.service.validate()
})
}
}
const normalizeResources = config => {
if (!config.resources) {
return config.resources
}
if (!config.resources.Resources) {
return {}
}
if (!Array.isArray(config.resources.Resources)) {
return config.resources
}
const newResources = config.resources.Resources.reduce(
(sum, { Resources, Outputs = {} }) => ({
...sum,
...Resources,
Outputs: {
...(sum.Outputs || {}),
...Outputs
}
}),
{}
)
return {
Resources: newResources
}
}
export async function loadServerlessConfig(cwd = process.cwd()) {
const stat = fs.statSync(cwd)
if (!stat.isDirectory()) {
cwd = path.dirname(cwd)
}
const serverless = new ConfigServerless()
await serverless.getConfig(cwd)
const { service: config } = serverless
const { custom = {} } = config
const output = {
...config,
custom: {
...custom
},
resources: normalizeResources(config)
}
return output
}