My Front Matter has the following Categories & Tags.
category:
- catA
- catB
- catC
tags:
- tagA
- tagB
- tagC
Middleman-Blog creates three separate folders for each tag under "Tags" folder. While expecting the same functionality for "category", it generates one folder named catA-catB-catC.
How do I generate three folders instead?
Related
I want to get competition.name from a list of submissions.
In my setup, competitions and teams share a M2M relationship (with an associated competition-team object. Each competition-team pair can submit any number of submissions. I now have a dashboard page which I am trying to create a table of all submissions by the team accompanied by the respective competition's name. The output should look like:
| Submission Name | Submission Date etc. | Competition Name |
| Sub01 | 2020-12-30 2000 | Competition01 |
I have trouble retrieving the competition name from the submissions. Here are my models:
class Competition(models.Model):
id = models.AutoField(primary_key=True)
name = models.CharField(max_length=30)
class CompetitionTeam(models.Model):
competition_id = models.ForeignKey('Competition', on_delete=models.CASCADE, to_field='id', db_column='competition_id')
team_id = models.ForeignKey('Team', on_delete=models.CASCADE, to_field='id', null=True, db_column='team_id')
class CompetitionSubmission(models.Model):
competitionteam_id = models.ForeignKey(CompetitionTeam, on_delete=models.CASCADE, db_column='competitionteam_id')
I wish to annotate a set of submissions with their respective competition names. I tried with:
submissions.annotate(competition_name=Subquery(Competition.objects.filter(id=Subquery(CompetitionTeam.objects.get(id=OuterRef('competitionteam_id')).competition_id)).values('name')))
"ValueError: This queryset contains a reference to an outer query and may only be used in a subquery."
I also tested with the following command:
CompetitionSubmission.objects.prefetch_related('competitionteam_id__competition_id')
It runs but the command seems to do nothing. I will update this post with other methods I try.
Thank you.
EDIT
submissions.annotate(competition_name=Subquery(Competition.objects.filter(id=Subquery(CompetitionTeam.objects.filter(id=OuterRef(OuterRef('competitionteam_id_id'))).values('competition_id'))).values('name')))
Seems to work correctly.
You can traverse ForeignKeys directly using double underscores.
CompetitionSubmission.objects.values(
'competitionteam_id',
'competitionteam_id__competition_id',
'competitionteam_id__competition_id__name',
'competitionteam_id__team_id',
'competitionteam_id__team_id__name',
)
This will only produce a single database query. Django ORM takes care of everything.
P.S. I would avoid using '_id' in field names as Django model fields are supposed to be referring to related objects themselves. Django automatically adds extra attributes with '_id' that contains the related object's id. Please see https://docs.djangoproject.com/en/3.1/ref/models/fields/#database-representation
How do I refer to relative path from view1 to view2 using ref('package_name', 'model_name') ?
/root_folder
/ project1
/models
view1.sql
dbt_project.yml
/ project2
/models
view2.sql
dbt_project.yml
There is no code example in the documentation.
Thank you.
The only way for project2 to know about models in project1 is if project2 included project1 as a package in its packages.yml file. Then you could refer to view1 as ref('project1', 'view1') in project2.
You'll have to check the syntax here, but you could include project1 in project2 in packages.yml like so:
in project2\packages.yml:
packages:
- local: ../project1
Needless to say, you'd save yourself a lot of headaches by simply not splitting projects. In most cases you shouldn't need to do that and simple folderization does most of what you might need.
To try and answer the question with a little more focus on your comment:
I have one folder for each dataset I have on bigquery. Can I write models for several datasets in one single folder?
Yes you can!
Quick note on terms from the dbt docs "BigQuery configurations" in case you are not using bigquery.
schema is interchangeable with the BigQuery concept dataset
database is interchangeable with the BigQuery concept of project
Here is how this works for me:
project-dir
| analysis
| data
| macros
| models
|> sources
- dataset1.yml
- dataset2.yml
| seed
| dbt_project.yml
| packages.yml
Where the contents of a dataset.yml is:
version: 2
sources:
- name: fivetran_log
database: my-bigquery-project-id
loader: fivetran
tables:
- name: account
- name: log
- name: user
No references are required within the dbt_project.yml to utilize these sources immediately. Instead, you can reference this directly from models like:
select *
from {{ source('fivetran_log', 'user') }}
That should allow you to have multiple dataset sources, but one single dbt project directory for all your views.
However, if the datasets you are referencing are within different bigquery regions or different billing projects, I believe you will run into some errors.
Appendix of related questions / resources across the dbt-verse:
Should I have an organisation wide project (a monorepo) or should each work flow have their own?
Building dbt models to be compatible with multiple data warehouses
I have a giant JSON array of every US state and city, along with other data about each of them. I'd like to iterate over the JSON and output a tree structure like this:
[Alabama]
index.html
[Abbeville]
index.html
[Adamsville]
index.html
[Alaska]
index.html
[Anchorage]
index.html
[Fairbanks]
index.html
...etc
I'd have two layouts:
state.html
city.html
So far, I haven't found a great way to do this. A lot of static gens seem to have the ability to use JSON for meta data within content, but not for the primary source of content.
thanks!
The Middleman static site generator supports this. You use their dynamic pages to create a list of the pages to create. The data comes from their data file feature. Here are the links to the pages explaining these
https://middlemanapp.com/advanced/dynamic_pages/
https://middlemanapp.com/advanced/data_files/
You would do something like have states.yml containing the state and city data
- states
- name: Alabama
cities:
- name: Abbeville
pop: X
- name: Adamsville
pop: Y
and then create the proxy pages e.g.
data.states.each do |state|
proxy "/#{state.name}/index.html", "templates/state.html", :locals => {state: state}
state.cities.each do |city|
proxy "/#{state.name}/#{city.name}/index.html", "/templates/city.html", :locals => {state: state, city: city}
end
end
The documentation describes a meta collection as a combination of 2 (or more) different APIs...
By default, a meta API endpoint returns all data from all of its
sub-APIs, organized by collection.
But, what I find is that it does not do this. It grabs partial data from some sub-APIs and fails silently, without logging an error.
API #1 Result
COLLECTION2
ROW LIST.HREF LIST.TEXT INDEX URL
1 http://www.amazon.com/Org...pollux+organix+canned+dog Canned Dog Food 3 http://www.austinpetsalive.org/donate/wish-list/
2 http://www.amazon.com/Pre...s-Large/lm/R34ISSXSRJPA71 Premier Brand Martingale collars (Pink, Silver, Blue, Red, and Orange) sizes XL or L 4 http://www.austinpetsalive.org/donate/wish-list/
3 http://www.amazon.com/Pre...ords=medium+gentle+leader Medium & Large Gentle Leaders 5 http://www.austinpetsalive.org/donate/wish-list/
API #2 Result
COLLECTION2
ROW LIST.HREF LIST.TEXT INDEX URL
1 Dry kitten and adult dog food (we ask that the first ingredient listed on the bag be meat). Some Brands we love include: Purina One, Pro-Plan, Wellness, Evolve, Blue Buffalo, and Kirklands 9 http://pawsshelter.org/donate/wishlist/
2 Cat Litter 10 http://pawsshelter.org/donate/wishlist/
3 Kongs, Balls, Durable Toys, Puzzle Toys 11 http://pawsshelter.org/donate/wishlist/
Meta API Result Collection 2 -- it's the entire data set from API #2 and none from API #1
COLLECTION2
ROW LIST.HREF LIST.TEXT INDEX URL API
1 Dry kitten and adult dog food (we ask that the first ingredient listed on the bag be meat). Some Brands we love include: Purina One, Pro-Plan, Wellness, Evolve, Blue Buffalo, and Kirklands 132 http://pawsshelter.org/donate/wishlist/ PAWS Shelter and Humane Society
2 Cat Litter 133 http://pawsshelter.org/donate/wishlist/ PAWS Shelter and Humane Society
3 Kongs, Balls, Durable Toys, Puzzle Toys 134 http://pawsshelter.org/donate/wishlist/ PAWS Shelter and Humane Society
The data structure is verbatim the same. Why don't they combine under meta API?
Testing
A meta API combination worked with 2 APIs, each with one collection.
When there is more than one collection defined, certain combinations of APIS combined partially. Each collection filled with the contents of one or the other API, but never both.
The 3 APIs I need to combine result in one site's results in all of the collections, and nothing from the other two in any collection, and no error logged.
The flaky results seem tied to the collections. But I've moved on to a klugey temp fix, which is to call each API separately and combine the JSON results in my app. 3 API calls instead of one. Might be looking for another tool, soon.
Now it works
Since I posted this question in October, KimonoLabs has updated their app and I am now having consistent success with creating and using a Meta API for the set of single APIs I posted above.
That's the positive.
The catch is that you cannot use the URL parameters in a meta API, so if you'd written post-process modify results functions, they will not be executed, even if you use kimmodify=1.
I have a site where I'm making URLs look like the following (for users to view information):
/:state/:data_type/:category/:subcategory/:item_id
data_type is one of "directory", "articles", or "documents". The idea is to store information for each state, based on categories and subcategories, and viewable as such.
Any particular piece of that is a valid path. For instance:
/IN - show a home page for Indiana (controller region_home, action index)
/IN/directory - list of categories and subcategories for Indiana (controller region_categories)
/IN/directory/schools - list of subcategories under the "schools" category for Indiana (controller is region_subcategories)
/IN/directory/schools/elementary - list of elementary schools in Indiana (controller is region_directory, action index)
/IN/directory/schools/elementary/10-meridian-street - drilled down to the specific school (controller is region_directory, action is show)
This is mainly for showing the information. I already have "slugs" created for all the regions, categories, subcategories, etc. That part works.
I can actually make it work like this:
match ':region_slug/directory(/:category_slug(/:subcategory_slug(/:directory_entry_id)))' => 'region_directory#index', :as => :region_directory
The issue is that every request goes to the same controller/action, which then has to dispatch the request.
I can cut it into pieces:
match ':region_slug' => 'region_home#index'
match ':region_slug/directory' => 'region_categories#index'
match ':region_slug/directory/:category_slug' => 'region_subcategories#index'
match ':region_slug/directory/:category_slug/:subcategory_slug' => 'region_directory#index'
match ':region_slug/directory/:category_slug/:subcategory_slug/:directory_entry_id' => 'region_directory#show'
But that breaks DRY, and I have to repeat that for articles, etc. I can't figure out how to do this with scopes. Any ideas?