Pretty new to Azure and struggling with creating a VM from an existing vhd. I get the following error when executing New-AzureQuickVM -ImageName MyVirtualHD.vhd -Windows -ServiceName test:
CurrentStorageAccountName is not accessible. Ensure that current storage account is accessible and the same location or affinity group as your cloud service.
Select-AzureRMSubscription does not return anything for the CurrentStorageAccount property. Get-AzureRMStorageAccount does list my storage account.
Azure has two deployment models: "Classic" and "Resource Manager" (ARM). You're not seeing your ARM-created storage accounts because you're using classic-mode powershell commands to list storage accounts, and your storage accounts were created with the (newer) Resource Management API (and the classic API will only list storage accounts created with the "classic" management API).
Your example shows you mixing the two types. (also - don't worry about resource groups in this context - that's not your issue - resource groups are unrelated for this).
Once you select your subscription (via Select-AzureRmSubscription), and then Get-AzureRmStorageAccount, you should see all of your newly created storage accounts.
Also: Set-AzureSubscription does something different - it's for altering subscription properties. You want Select-... for selecting the default subscription to work with.
Related
We are setting up an active/active configuration using either front door or traffic manager as our front end. Our services are located in both Central and East US 2 paired regions. There is an AKS cluster in each region. The AKS clusters will write data to a storage account located in their region. However, the files in the storage accounts must be the same in each region. The storage accounts must be zone redundant and read/writeable in each region at all times, thus none of the Microsoft replication strategies work. This replication must be automatic, we can't have any manual process to do the copy. I looked at Data Factory but it seems to be regional, so I don't think that would work, but it's a possibility....maybe. Does anyone have any suggestions on the best way to accomplish this task?
I have tested in my environment.
Replication between two storage accounts can be implemented using the Logic App.
In the logic app, we can create two workflows. One for replicating data from storage account 1 to storage account 2. Other for replicating data from storage account 2 to storage account 1.
I have tried to replicate blob data between storage accounts in different regions.
The workflow is :
When a blob is added or modified in the storage account 1, the blob will be copied to the storage account 2
Trigger : When a blob is added or modified (properties only) (V2) (Use connection setting of storage account1)
Action : Copy blob (V2) ) (Use connection setting of storage account2)
Similar way, we can create another workflow for replication of data from Storage Account 2 to Storage Account 1.
Now, the data will be replicated between the two storage accounts.
My team is planning on building a data processing pipeline that will involve S3 integration with Snowflake. This article from Snowflake shows that an AWS IAM role must be created in order for Snowflake to access S3's data.
However, in our pipeline, we need to ensure multi-tenancy and data isolation between users. For example, let's assume that Alice and Bob has files in S3 under "s3://bucket-alice/file_a.csv" and "s3://bucket-bob/file_b.csv" respectively. Then, we want to make sure that, when staging Alice's data onto Snowflake, Alice can only access "s3://bucket-alice" and nothing under "s3://bucket-bob". This means that individual AWS IAM roles must be created for each user.
I do realize that Snowflake has it's own access control system, but my team wants to make sure that data isolation is fully achieved from the S3-to-Snowflake stage of the pipeline, and not only relying on Snowflake's access control.
We are worried that this will not be scalable, as AWS sets a limit of 5000 IAM users, and that will not be enough as we scale our product. Is this the only way of ensuring data multi-tenancy, and does anyone have a real-world application example of something like this?
Have you explored leveraging Snowflake's Internal Stage, instead? By default, every user gets their own internal stage that only they have permissions to from within Snowflake and NO access outside of Snowflake. Snowflake offers the ability to move data in and out of that Internal Stage using just about every driver/connector that Snowflake has available. This said, any pipeline/workflow that is being leveraged by 5000+ users would be able to use these connectors to load data to Snowflake Internal Stage (S3) without the need for any additional AWS IAM Users. Would that be a sufficient solution for your situation?
I have an Azure storage account.
For a while now, something has been creating 4 character empty containers as shown here, there are hundreds of them:
This storage account is used by:
Function Apps
Document Db (Cosmos)
Terraform State
Container Registry for Docker images
It's not a big deal but I don't want millions of empty containers being created by an unknown process.
Note1: I have looked for any way to find more statistics / history of these folders but I cant find any
Note2: We don't have any custom code that creates storage containers in our release pipelines (ie... PowerShell or CLI)
thanks
Russ
It seems the containers are used to store logs of Azure Function. I have a storage account just for azure function and web app. We could see it has the containers like yours via Storage Explorer.
Is there a solution to get the below informations of Window File Azure storage Account using Windows Azure Storage Client Library:
Azure Storage Account Capacity
Azure Storage Free and used Space
Azure Storage Account State (Active, Disable, Enable ….)
Client Transfer files (Mo, GO … ) per month, days …
Azure Storage Account Performance
...
Thanks
As far as I know, a azure standard account contains multiple services. Blob, table, queue, file.
If you want to know the information about he file service, you could use Windows Azure Storage Client Library. If you want to know the information about your storage account, I suggest you could use azure management library.
Azure Storage Account Capacity
As far as I know, the azure storage account capacity is 500TB.
Max size of a file share is 5TB.
Max size of a file is 1TB.
We could create multiple file share in one storage account. The only limit is the 500 TB storage account capacity.
More details, you could refer to this article.
Azure Storage Free and used Space
As far as I know, we could only get the quota and usage of a fileshare by using the Windows Azure Storage Client Library.
We could use CloudFileShare.Properties.Quota property to get the quota of the fileshare and use CloudFileShare.GetStats method to get the usage of the fileshare.
More details, you could refer to below codes:
CloudStorageAccount storageAccount = CloudStorageAccount.Parse(
"connectionstring");
CloudFileClient fileClient = storageAccount.CreateCloudFileClient();
CloudFileShare share = fileClient.GetShareReference("fileshare");
share.FetchAttributes();
//get the quota
int? i = share.Properties.Quota;
//get usage
var re = share.GetStats();
Console.WriteLine(i);
Console.WriteLine(re.Usage);
Azure Storage Account State (Active, Disable, Enable ….)
As far as I know, we couldn't get storage account state by using storage SDK. If you want to get this value, I suggest you could use azure management library. You could install it from Nuget package. You could get the StorageAccount.Properties.Status from the StorageAccounts class.
More details about how to use azure management library to access the storage account you could refer to this article.
Client Transfer files (Mo, GO … ) per month, days …
As far as I know, the Windows Azure Storage Client Library doesn't contain the method to get the client transfer files (Mo, GO … ) per month, days.
Here is a workaround, you could write codes to calculate the transfer files number in your application and store this number to azure table storage per day.(When uploading the file to the azure file storage, firstly get the number from the table and add one, then upload the number to the table storage)
If you want to get the number of the transfer files, you could use the azure table storage SDK to get the result.
Azure Storage Account Performance
As far as I know, if we want to check our azure storage account performance, we should firstly enable the diagnostics to log how the storage works. Then we could check the storage performance by using its service's metrics.
More details about how to access metrics data by using Windows Azure Storage Client Library. I suggest you could refer to this article.
As per my understanding it is user who has access to publish the specific page/resource.
Documentation goes like this:
Depending on the environment, the agent will use this user account to:
collect and package the content from the author environment
create and write the content on the publish environment
Leave this field empty to use the system user account (the account defined in sling as the administrator user; by default this is admin).
means this replication agent comes into action only when replicating the content from packagemenager(by clicking replicate for specific package) ? or activating the page/resource from siteadmin?
The Agent User ID property is used to manage what part of content tree will be replicated using given replication queue. This has nothing to do with actual package creation - it applies to all replication process.
Multitenant use case
For the complicated infrastructure it may happen that the multi-tenant architecture involves some sharding approach. Imagine a geo-spread architecture with no CDN involved where the brand site should be quickly accessible from the given localisation. Due to technical limitations, pushing whole content (all sites) around the world might not be acceptable.
Dedicated DAM environment use case
When DAM storage is shared across multiple AEM implementations it is often desired to dettach that from the regular authoring by creating a separated DAM-only instance. On such platform the replication agents should be configured to have the read access to /content/dam only in order not to mess up with other content trees.
Solution
In this case, the user agent ID can be configured to use a dedicated user permission scheme. All the changes the preconfigured user sees will be replicated to the corresponding endpoint. There are technical alternatives like implementing a transport handler (see https://github.com/Cognifide/CQ-Transport-Handler/blob/master/README.md)