転移学習やファインチューニングをしたモデルでGrad-CAMをしたらValueError: Graph disconnected

転移学習やファインチューニングをしたモデルでGrad-CAMを試そうとしたところ、エラー発生。

構築したモデルはこれ。

base = VGG16(input_shape=(224, 224, 3), weights='imagenet', include_top=False)

inp = tf.keras.layers.Input(shape=(224, 224, 3))
x = base(inp)
x = tf.keras.layers.GlobalAveragePooling2D()(x)
x = tf.keras.layers.Dense(n_category, activation='softmax')(x)

model = tf.keras.Model(inputs=inp, outputs=x)

_________________________________________________________________
Layer (type)                 Output Shape              Param #
=================================================================
input (InputLayer)           (None, 224, 224, 3)       0
_________________________________________________________________
vgg16 (Functional)           (None, 7, 7, 512)         14714688
_________________________________________________________________
global_average_pooling2d (Gl (None, 512)               0
_________________________________________________________________
dense (Dense)                (None, 6)                 3078
=================================================================
Total params: 14,717,766
Trainable params: 14,717,766
Non-trainable params: 0
_________________________________________________________________

Grad-CAMに投げる時、最終層の畳み込みの重みと予測結果を同時に取得するためにFunctional Modelを構築する。

そして画像を入力すると表題のエラー。

grad_model = tf.keras.models.Model(inputs = [model.input], 
                                   outputs=[model.output, model.get_layer(LAYER_NAME).output])

grad_model(img)


ValueError: Graph disconnected: cannot obtain value for tensor 
Tensor("input_1:0", shape=(?, 224, 224, 3), dtype=float32) at layer "input_1". 
The following previous layers were accessed without issue: []

どうやらグラフの構築が上手く行っていないようで、inputがどこにも繋がってないと怒られる。

エラーを回避するために、Input Layerを介さずにベースモデル（VGG16）の入力を起点としてFunctional Modelを構築する。

tf.keras.models.Modelの引数をチェック。

Arguments
inputs : The input(s) of the model: a keras.Input object or list of keras.Input objects.
outputs : The output(s) of the model. See Functional API example below.
https://www.tensorflow.org/api_docs/python/tf/keras/Model

引数の条件を満たすようにFuncional Modelを書き換え。

base = VGG16(input_shape=(224, 224, 3), weights='imagenet', include_top=False)

x = base.output
x = tf.keras.layers.GlobalAveragePooling2D()(x)
x = tf.keras.layers.Dense(n_category, activation='softmax')(x)

model = tf.keras.Model(inputs=base.input, outputs=x)

_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
input_1 (InputLayer)         (None, 224, 224, 3)       0         
_________________________________________________________________
block1_conv1 (Conv2D)        (None, 224, 224, 64)      1792      
_________________________________________________________________
block1_conv2 (Conv2D)        (None, 224, 224, 64)      36928     
_________________________________________________________________
block1_pool (MaxPooling2D)   (None, 112, 112, 64)      0         
_________________________________________________________________
block2_conv1 (Conv2D)        (None, 112, 112, 128)     73856     
_________________________________________________________________
block2_conv2 (Conv2D)        (None, 112, 112, 128)     147584    
_________________________________________________________________
block2_pool (MaxPooling2D)   (None, 56, 56, 128)       0         
_________________________________________________________________
block3_conv1 (Conv2D)        (None, 56, 56, 256)       295168    
_________________________________________________________________
block3_conv2 (Conv2D)        (None, 56, 56, 256)       590080    
_________________________________________________________________
block3_conv3 (Conv2D)        (None, 56, 56, 256)       590080    
_________________________________________________________________
block3_pool (MaxPooling2D)   (None, 28, 28, 256)       0         
_________________________________________________________________
block4_conv1 (Conv2D)        (None, 28, 28, 512)       1180160   
_________________________________________________________________
block4_conv2 (Conv2D)        (None, 28, 28, 512)       2359808   
_________________________________________________________________
block4_conv3 (Conv2D)        (None, 28, 28, 512)       2359808   
_________________________________________________________________
block4_pool (MaxPooling2D)   (None, 14, 14, 512)       0         
_________________________________________________________________
block5_conv1 (Conv2D)        (None, 14, 14, 512)       2359808   
_________________________________________________________________
block5_conv2 (Conv2D)        (None, 14, 14, 512)       2359808   
_________________________________________________________________
block5_conv3 (Conv2D)        (None, 14, 14, 512)       2359808   
_________________________________________________________________
block5_pool (MaxPooling2D)   (None, 7, 7, 512)         0         
_________________________________________________________________
global_average_pooling2d (Gl (None, 512)               0         
_________________________________________________________________
dense (Dense)                (None, 6)                 3078      
=================================================================
Total params: 14,717,766
Trainable params: 14,717,766
Non-trainable params: 0
_________________________________________________________________

これでエラーが出なくなった。

転移学習やファインチューニング、その他独自のモデルで試しても、Grad−CAMでエラーが出なくなり満足。

参考

Gradcam with guided backprop for transfer learning in Tensorflow 2.0

I get an error using gradient visualization with transfer learning in TF 2.0. The gradient visualization works on a mode...

Google Colab

GitHub - sicara/tf-explain: Interpretability Methods for tf.keras models with Tensorflow 2.x

Interpretability Methods for tf.keras models with Tensorflow 2.x - sicara/tf-explain