Overlapping ranks - Creating a block diagram - block

I nearly spent the whole day trying to get a block diagram out of Graphviz. The biggest problem seems to be that I've got vertical as well as horizontal ranking. Easier shown than explained, so here is how the diagram should look:
This is my Dot-Code so far:
digraph block_diagram
{
graph [ splines=ortho, rankdir=TB];
node [shape=box, style=filled, fontname=Arial];
layer_0 [style="rounded,filled", fillcolor="#99ccff", width=15, height=1, label="Layer 0"];
layer_1 [shape=box, width=5, height=1, label="Layer 1"];
//emptybox [shape=box,height=1, label=""];
layer_1_1 [shape=box, width=5, height=1, label="Layer 1_1"];
layer_2 [shape=box, width=5, height=1, label="Layer 2"];
layer_3 [shape=box, width=5, height=1, label="Layer 3"];
layer_4 [shape=box, width=5, height=1, label="Layer 4"];
application [shape=box, width=3, height=5, label="Application"];
layer_0 -> { layer_1 application };
layer_0 -> layer_1 -> layer_1_1 -> layer_2 -> layer_3 -> layer_4;
application -> { layer_2 layer_3 layer_4 };
}
It looks like this -
I tried to group the nodes and rank them, but in the end all have the same rank and the diagram just looks wrong.
I don't know what to do anymore, and I hope someone can help me.
P.S.: I also don't know to create a shape like 'Layer 1'

Summary: This answer presents a variant using auxiliary nodes, still looking for a variant with no extra nodes and clusters.
If you create several invisible nodes within a cluster, you can get desired visual behavior, as in this image:
This is possible through the use of the attribute newrank=true to graph and { rank=same; nodename1; nodename2; ... } for horizontal alignment desired nodes. To hide auxiliary nodes, you need to give them an attribute style=invis, to hide cluster's borders add attribute color=none to cluster.
For vertical alignment, group the desired nodes by giving them an attribute group=g1, here g1 is your custom name for column of nodes.
Layer 1 and Layer 1_1 nodes can be created as table cells.
Bi-directional edges created with dir=both.
To make the line end at the cluster boundary, I used attribute compound=true for graph and lhead, ltail for edge.
Final image:
Script:
digraph block_diagram {
graph [
newrank=true,
ranksep=0,
nodesep=0,
splines=ortho,
compound=true,
fontname="Arial" ];
node [
shape=box,
style=filled,
fontname="Arial",
fontsize=14 ];
subgraph cluster_0{
fillcolor="#99ccff"
style="rounded, filled"
layer_0 [
label=<<B>Layer 0</B>>,
fillcolor=none,
shape=none
width=1 ];
a1 [group=g1 width=1.5 style=invis]
a2 [group=g2 width=1.5 style=invis]
}
subgraph cluster_1_1 {
color=none;
layer_1 [
group=g1,
fillcolor=none,
shape=none,
margin=0,
label= <<table border="0" cellspacing="0" cellborder="1">
<tr>
<td width="40" height="30" fixedsize="true" BGCOLOR="#ccffff" sides="TL"></td>
<td width="80" height="30" fixedsize="true" BGCOLOR="#ccffff" sides="TRB" ALIGN="LEFT"><B>Layer 1</B></td>
</tr>
<tr>
<td width="40" height="30" fixedsize="true" BGCOLOR="#ccffff" sides="RBL"></td>
<td COLSPAN="2" width="72" height="26" fixedsize="true" BGCOLOR="#ccffff" sides="TBRL" VALIGN="BOTTOM"><B>Layer 1_1</B></td>
</tr></table>>]
layer_2 [
group=g1,
label=<<B>Layer 2</B>>,
fillcolor ="#ccffff",
width=1.6 ];
}
layer_underline [
group=g1,
label="",
margin=0;
shape=underline,
fillcolor=none,
width=2,
height=0 ];
subgraph cluster_1_2{
color=none;
layer_3 [
group=g1,
label=<<B>Layer 3</B>>,
fillcolor="#ccff99",
width=1.6 ];
layer_4 [
group=g1,
label=<<B>Layer 4</B>>,
fillcolor="#ccff99",
width=1.6 ];
}
subgraph cluster_2 {
style=filled;
fillcolor="#dde6ed"
b1 [group=g2 style=invis]
application [
group=g2,
label=<<B>Application</B>>,
shape=none
fillcolor=none
width=2 ];
b2 [group=g2 style=invis]
}
a1 -> layer_0 [style=invis]
layer_0 -> a2 [style=invis]
a1 -> layer_1 [minlen=3 ltail="cluster_0" lhead="cluster_1_1" dir="both" color="#99ccff" penwidth=5 arrowsize=.5]
a2 -> b1 [ltail="cluster_0" lhead="cluster_2" dir=both color="#99ccff" penwidth=5 arrowsize=.5]
layer_1 -> layer_2 [style=invis];
layer_2 -> layer_underline [ color="#99ccff" penwidth=5 arrowsize=.5 dir=back ltail="cluster_1_1"]
layer_underline -> layer_3 [ color="#99ccff" penwidth=5 arrowsize=.5 lhead="cluster_1_2"]
layer_3 -> layer_4 [style=invis]
b1 -> application [style=invis]
application -> b2 [style=invis]
layer_4 -> b2 [color="#99ccff" penwidth=5 arrowsize=.5 ltail="cluster_1_2" lhead="cluster_2" dir=both]
layer_2 -> application [color="#99ccff" penwidth=5 arrowsize=.5 ltail="cluster_1_1" lhead="cluster_2" dir=both]
{ rank=same; layer_4; b2 }
{ rank=same; layer_1; b1 }
{ rank=same; layer_underline; application }
{ rank=same; a1; layer_0; a2 }
}

Related

How to evaluate the multi-class classifier prediction with TensorFlow Model Analysis library?

I'm trying to use TensorFlow Model Analysis library to analyze the prediction data from a multi-class classifier model, using the analyze_raw_data API. Currently the label contains 3 different classes [0, 1, 2], trained with SparseCategoricalCrossentropy loss. The tfma config was set as following:
eval_config = text_format.Parse("""
## Model information
model_specs {
label_key: "label",
prediction_key: "predictions"
}
## Post training metric information. These will be merged with any built-in
## metrics from training.
metrics_specs {
metrics { class_name: "ExampleCount" }
metrics { class_name: "SparseCategoricalCrossentropy" }
metrics { class_name: "SparseCategoricalAccuracy" }
metrics { class_name: "Precision" config: '"top_k": 1' }
metrics { class_name: "Precision" config: '"top_k": 3' }
metrics { class_name: "Recall" config: '"top_k": 1' }
metrics { class_name: "Recall" config: '"top_k": 3' }
metrics { class_name: "MultiClassConfusionMatrixPlot" }
}
## Slicing information
slicing_specs {} # overall slice
""", tfma.EvalConfig())
I've added ground truth label as a numerical value from [0, 1, 2] to the dataframe column "label", and predicted probabilities as a list to another column "predictions" (e.g. [0.2, 0.3, 0.5]), but I observed the error like ArrowTypeError: ("Expected bytes, got a 'numpy.ndarray' object", 'Conversion failed for column predictions with type object') when loading the data:
~/.pyenv/versions/3.7.8/lib/python3.7/site-packages/tensorflow_model_analysis/api/model_eval_lib.py in analyze_raw_data(data, eval_config, output_path, add_metric_callbacks)
1511
1512 arrow_data = table_util.CanonicalizeRecordBatch(
-> 1513 table_util.DataFrameToRecordBatch(data))
1514 beam_data = beam.Create([arrow_data])
1515
Does anyone know how to write the data for labels and predictions of multi-class classification so that we can do the model analysis with tfma?

Tensorflow Dataset operation equal to timeseries_dataset_from_array possible?

I want some more control over the TensorFlow dataset generation. For this reason, I want to mirror the behavior of timeseries_dataset_from_array but with the ability to use consecutive windows or non-overlapping windows (not possible with timeseries_dataset_from_array to set sequence_stride=0).
# df_with_inputs = (x, 19) df_with_labels = (x,1)
ds = tf.data.Dataset.from_tensor_slices((df_with_inputs.values, df_with_labels.values)).window(20, shift=1, stride=1, drop_remainder=True).batch(32)
equals to:
ds = tf.keras.preprocessing.timeseries_dataset_from_array(df_with_inputs[df_with_inputs.columns], df_with_labels[df_with_labels.columns], sequence_length=window_size,sequence_stride=1,shuffle=False,batch_size=batch_size)
both create a BatchDataset with the same amount of samples, but the type-spec of the dataset with the manual method is somehow different, e.g., first, give me:
<BatchDataset shapes: (DatasetSpec(TensorSpec(shape=(19,), dtype=tf.float32, name=None), TensorShape([None])), DatasetSpec(TensorSpec(shape=(1,), dtype=tf.float32, name=None), TensorShape([None]))), types: (DatasetSpec(TensorSpec(shape=(19,), dtype=tf.float32, name=None), TensorShape([None])), DatasetSpec(TensorSpec(shape=(1,), dtype=tf.float32, name=None), TensorShape([None])))>
where the last one give me:
<BatchDataset shapes: ((None, None, 19), (None, 1)), types: (tf.float64, tf.int32)>
. But both contain the same amount of elements, in my case, 3063. Note that stride and sequence_stride have different behavior in both methods (for the same behavior, you need shift=1). Additionally, when I try to feed the first to my NN, I receive the following error (where the ds of timeseries_dataset_from_array works like a charm):
TypeError: Inputs to a layer should be tensors.
Any idea what I am missing here?
My model:
input_shape = (window_size, num_features) #(20,19)
model = tf.keras.Sequential([
tf.keras.layers.Conv1D(filters=64, kernel_size=3, activation='relu', padding="same",
input_shape=input_shape), [....]])
The equivalent of this:
import tensorflow as tf
tf.random.set_seed(345)
samples = 30
df_with_inputs = tf.random.normal((samples, 2), dtype=tf.float32)
df_with_labels = tf.random.uniform((samples, 1), maxval=2, dtype=tf.int32)
batch_size = 2
window_size = 20
ds1 = tf.keras.preprocessing.timeseries_dataset_from_array(df_with_inputs, df_with_labels, sequence_length=window_size,sequence_stride=1,shuffle=False, batch_size=batch_size)
for x, y in ds1.take(1):
print(x, y)
tf.Tensor(
[[[-0.01898661 1.2348452 ]
[-0.33379436 -0.13637085]
[-2.239644 1.5407541 ]
[-0.14988706 0.50577176]
[-1.6328571 -0.9512018 ]
[-3.0481005 0.8019097 ]
[-0.683125 -0.12166552]
[-0.5408724 -0.97584397]
[ 0.47595206 1.0512688 ]
[ 0.15297593 0.7393363 ]
[-0.17052855 -0.12541457]
[ 1.1617764 -2.491248 ]
[-2.5665069 0.9241422 ]
[ 0.40681016 -1.031384 ]
[-0.23945935 1.5275828 ]
[-1.3431666 0.2940185 ]
[ 1.7351524 0.34276873]
[ 0.8059861 2.0647929 ]
[-0.3017126 0.729208 ]
[-0.8672192 -0.79938954]]
[[-0.33379436 -0.13637085]
[-2.239644 1.5407541 ]
[-0.14988706 0.50577176]
[-1.6328571 -0.9512018 ]
[-3.0481005 0.8019097 ]
[-0.683125 -0.12166552]
[-0.5408724 -0.97584397]
[ 0.47595206 1.0512688 ]
[ 0.15297593 0.7393363 ]
[-0.17052855 -0.12541457]
[ 1.1617764 -2.491248 ]
[-2.5665069 0.9241422 ]
[ 0.40681016 -1.031384 ]
[-0.23945935 1.5275828 ]
[-1.3431666 0.2940185 ]
[ 1.7351524 0.34276873]
[ 0.8059861 2.0647929 ]
[-0.3017126 0.729208 ]
[-0.8672192 -0.79938954]
[-0.14423785 0.95039433]]], shape=(2, 20, 2), dtype=float32) tf.Tensor(
[[1]
[1]], shape=(2, 1), dtype=int32)
Using tf.data.Dataset.from_tensor_slices would be this:
ds2 = tf.data.Dataset.from_tensor_slices((df_with_inputs, df_with_labels)).batch(batch_size)
inputs_only_ds = ds2.map(lambda x, y: x)
inputs_only_ds = inputs_only_ds.flat_map(tf.data.Dataset.from_tensor_slices).window(window_size, shift=1, stride=1, drop_remainder=True).flat_map(lambda x: x.batch(window_size)).batch(batch_size)
ds2 = tf.data.Dataset.zip((inputs_only_ds, ds2.map(lambda x, y: y)))
for x, y in ds2.take(1):
print(x, y)
tf.Tensor(
[[[-0.01898661 1.2348452 ]
[-0.33379436 -0.13637085]
[-2.239644 1.5407541 ]
[-0.14988706 0.50577176]
[-1.6328571 -0.9512018 ]
[-3.0481005 0.8019097 ]
[-0.683125 -0.12166552]
[-0.5408724 -0.97584397]
[ 0.47595206 1.0512688 ]
[ 0.15297593 0.7393363 ]
[-0.17052855 -0.12541457]
[ 1.1617764 -2.491248 ]
[-2.5665069 0.9241422 ]
[ 0.40681016 -1.031384 ]
[-0.23945935 1.5275828 ]
[-1.3431666 0.2940185 ]
[ 1.7351524 0.34276873]
[ 0.8059861 2.0647929 ]
[-0.3017126 0.729208 ]
[-0.8672192 -0.79938954]]
[[-0.33379436 -0.13637085]
[-2.239644 1.5407541 ]
[-0.14988706 0.50577176]
[-1.6328571 -0.9512018 ]
[-3.0481005 0.8019097 ]
[-0.683125 -0.12166552]
[-0.5408724 -0.97584397]
[ 0.47595206 1.0512688 ]
[ 0.15297593 0.7393363 ]
[-0.17052855 -0.12541457]
[ 1.1617764 -2.491248 ]
[-2.5665069 0.9241422 ]
[ 0.40681016 -1.031384 ]
[-0.23945935 1.5275828 ]
[-1.3431666 0.2940185 ]
[ 1.7351524 0.34276873]
[ 0.8059861 2.0647929 ]
[-0.3017126 0.729208 ]
[-0.8672192 -0.79938954]
[-0.14423785 0.95039433]]], shape=(2, 20, 2), dtype=float32) tf.Tensor(
[[1]
[1]], shape=(2, 1), dtype=int32)
Note that flap_map is necessary to flatten the tensor in order to apply sliding windows more easily. The function flat_map(lambda x: x.batch(window_size)) simply creates batches of the flattened tensor after applying sliding windows.
With the line inputs_only_ds = ds2.map(lambda x, y: x) I extract only the data (x) without the labels (y) to run sliding windows. Afterwards, in tf.data.Dataset.zip((inputs_only_ds, ds2.map(lambda x, y: y))), I concatenate / zip the dataset with the sliding windows and the labels (y) resulting in the final result ds2.

Output probability of prediction in tensorflow.js

I have a model.json generated from tensorflow via tensorflow.js coverter
In the original implementation of model in tensorflow in python, it is built like this:
model = models.Sequential([
base_model,
layers.Dropout(0.2),
layers.Flatten(),
layers.Dense(128, activation='relu'),
layers.Dense(num_classes)
])
In tensorflow, the probability can be generated by score = tf.nn.softmax(predictions[0]), according to the tutorial on official website.
How do I get this probability in tensorflow.js?
I have copied the codes template as below:
$("#predict-button").click(async function () {
if (!modelLoaded) { alert("The model must be loaded first"); return; }
if (!imageLoaded) { alert("Please select an image first"); return; }
let image = $('#selected-image').get(0);
// Pre-process the image
console.log( "Loading image..." );
let tensor = tf.browser.fromPixels(image, 3)
.resizeNearestNeighbor([224, 224]) // change the image size
.expandDims()
.toFloat()
// RGB -> BGR
let predictions = await model.predict(tensor).data();
console.log(predictions);
let top5 = Array.from(predictions)
.map(function (p, i) { // this is Array.map
return {
probability: p,
className: TARGET_CLASSES[i] // we are selecting the value from the obj
};
}).sort(function (a, b) {
return b.probability - a.probability;
}).slice(0, 2);
console.log(top5);
$("#prediction-list").empty();
top5.forEach(function (p) {
$("#prediction-list").append(`<li>${p.className}: ${p.probability.toFixed(6)}</li>`);
});
How should I modify the above code?
The output is just the same as the value of variable 'predictions':
Float32Array(5)
0: -2.5525975227355957
1: 7.398464679718018
2: -3.252196788787842
3: 4.710395812988281
4: -4.636396408081055
buffer: (...)
byteLength: (...)
byteOffset: (...)
length: (...)
Symbol(Symbol.toStringTag): (...)
__proto__: TypedArray
0: {probability: 7.398464679718018, className: "Sunflower"}
1: {probability: 4.710395812988281, className: "Rose"}
length: 2
__proto__: Array(0)
Please help!!!
Thanks!
In order to extract the probabilities from the logits of the model using a softmax function you can do the following:
This is the array of logits that are also the predictions you get from the model
const logits = [-2.5525975227355957, 7.398464679718018, -3.252196788787842, 4.710395812988281, -4.636396408081055]
You can call tf.softmax() on the array of values
const probabilities = tf.softmax(logits)
Result:
[0.0000446, 0.9362511, 0.0000222, 0.0636765, 0.0000056]
Then if you wanted to get the index with the highest probability you can make use of tf.argMax():
const results = tf.argMax(probabilities).dataSync()[0]
Result:
1
Edit
I am not too familiar with jQuery so this might not be correct. But here is how I would get the probabilities of the outputs in descending order:
let probabilities = tf.softmax(predictions).dataSync();
$("#prediction-list").empty();
probabilities.forEach(function(p, i) {
$("#prediction-list").append(
`<li>${TARGET_CLASSES[i]}: ${p.toFixed(6)}</li>`
);
});

Error: Size(XX) must match the product of shape x,x,x,x

This is a newbie question, but any help will be appreciated.
I'm having a problem with a 3D tensor in TensorFlow.JS (node), with the following code:
const tf = require('#tensorflow/tfjs-node');
(async ()=>{
let list = [
{
xs: [
[
[ 0.7910133603149169, 0.7923634491520086, 0.79166712455722, 0.7928027625311359, 0.4426631841175303, 0.018719529693542337 ],
[ 0.7890709817505044, 0.7943561081665688, 0.7915865358198619, 0.7905450669351226, 0.4413258183256521, 0.04449784810703526 ],
[ 0.7940229392692819, 0.7924745639669473, 0.7881395357356101, 0.7880208892359736, 0.40902353356570315, 0.14643954229459097 ],
[ 0.801474878324385, 0.8003822349633881, 0.7969969705961001, 0.7939094034872144, 0.40227041242732126, 0.03893523221469505 ],
[ 0.8022503526561848, 0.8011600386679555, 0.7974621873981194, 0.8011488339557422, 0.43008361179994464, 0.11210020422004835 ],
],
[
[ 0.8034111510684465, 0.7985390234525179, 0.7949321830852709, 0.7943788081438548, 0.5739870761673189, 0.13358267460835263 ],
[ 0.805714476773561, 0.8072996569653942, 0.8040745782073486, 0.8035592212810225, 0.5899031300445114, 0.03229758335964042 ],
[ 0.8103322733081704, 0.8114317495511435, 0.8073606480159334, 0.8057140734135828, 0.5842202187553198, 0.01986941729798157 ],
[ 0.815132106874313, 0.8122641403791668, 0.8104353115275772, 0.8103395749739932, 0.5838313552472632, 0.03332674037143093 ],
[ 0.8118480102237944, 0.8166500561770489, 0.8128943005604122, 0.8147644523703373, 0.601619389872815, 0.04807286626501376 ],
]
],
ys: 1
}
];
const ds = tf.data.generator(async () => {
let index = 0;
return {
next: async () => {
if(index >= list.length) return { done : true };
let doc = list[index];
index++;
return {
value: {
xs : doc.xs,
ys : doc.ys
},
done: false
};
}
};
}).batch(1);
let model = tf.sequential();
model.add(tf.layers.dense({units: 60, activation: 'relu', inputShape: [2, 5, 6]}));
model.compile({
optimizer: tf.train.adam(),
loss: 'sparseCategoricalCrossentropy',
metrics: ['accuracy']
});
await model.fitDataset(ds, {epochs: 1});
return true;
})().then(console.log).catch(console.error);
This code generate the following error:
Error: Size(60) must match the product of shape 1,2,5,60
at Object.inferFromImplicitShape
I didn't understand why the layer is changing the last value of the inputShape from 6 to 60 (which is the expected output units for this layer).
Just to confirm, as far I know the units should be the product of: batchSize * x * y * z, in the example case: 1 * 2 * 5 * 6 = 60
Thank you!
Software specification:
tfjs-node: v1.2.11
Node JS: v11.2.0
OS: Ubuntu 18.04.2
Ok, the problem is that a fully connected layer (ts.layer.dense) expect a tensor1d as input, as described in this other question: Why do we flatten the data before we feed it into tensorflow?
So, to do the trick, the tensor must be re-shaped before the fully connected layer, as:
return {
value: {
xs : ts.reshape(doc.xs, [-1]),
ys : doc.ys
},
done: false
};
Where the -1 in ts.reshape(tensor, [-1]), means to the transformation function flatten the tensor.
For a visual demonstration, here a YouTube video: CNN Flatten Operation Visualized

How can i use 38 classes instead of 1000 in model.predict decode predictions

i am founding an error in plant disease detection using resnet50 deep learning model every time it raises an error message in decode_predictions
error
expects a batch of predictions (i.e. a 2D array of shape (samples, 1000)). Found array with shape: (1, 38)"
enter code here
model = ResNet50(weights='imagenet',include_top=False,classes=38)
try:
model = load_model('/content/drive/My
Drive/color/checkpoints/ResNet50_model_weights.h5')
print("model loaded")
except:
print("model not loaded")
img_path = '/content/drive/My Drive/color/test/0/appleblackrot188.jpg'
img = image.load_img(img_path, target_size=(300, 300))
x = image.img_to_array(img)
x = np.expand_dims(x, axis=0)
x = preprocess_input(x)
preds = model.predict(x)
print('Predicted:', decode_predictions(preds,top=3)[0])
You can try using the preprocesing function:
import tensorflow as tf
# Using the keras wrapper on tensorflow (it must be the same using just keras).
IMAGE = [] # From image source, i did it from the camera.
toPred = tf.keras.applications.resnet50.preprocess_input(np.expand_dims(tf.keras.preprocessing.image.img_to_array(IMAGE), axis=0))
Maybe that can help :)
decode_predictions works only for ImageNet (no. of classes = 1000). For these 38 classes of plants, you have to write your own decode predictions based on the ground truth label you've assigned for each plant.
First, you need an index JSON file and create a new decode_predictions function.
For example
This HAM10000 that has 7 classes and you need to split to each folder like this
then make an index JSON file like this
{
"0" : [
"AKIEC",
"akiec"
],
"1" : [
"BCC",
"bcc"
],
"2" : [
"BKL",
"bkl"
],
"3" : [
"DF",
"df"
],
"4" : [
"MEL",
"mel"
],
"5" : [
"NV",
"nv"
],
"6" : [
"VASC",
"vasc"
]
}
Then try this code
def decode_predictions(preds, top=4, class_list_path='/content/SKIN_DATA/HAM10000_index.json'):
if len(preds.shape) != 2 or preds.shape[1] != 7: # your classes number
raise ValueError('`decode_predictions` expects '
'a batch of predictions '
'(i.e. a 2D array of shape (samples, 1000)). '
'Found array with shape: ' + str(preds.shape))
index_list = json.load(open(class_list_path))
results = []
for pred in preds:
top_indices = pred.argsort()[-top:][::-1]
result = [tuple(index_list[str(i)]) + (pred[i],) for i in top_indices]
result.sort(key=lambda x: x[2], reverse=True)
results.append(result)
return results