kerasVGG16を使って全結合層の4096次元特徴量を抽出したい

Question

前提・実現したいこと

http://testpy.hatenablog.com/entry/2017/07/22/195400
↑を参考に、Imagenetで学習済みのVGG16を使って、フォルダ内画像の特徴量を抽出したい。

発生している問題

_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
input_3 (InputLayer)         (None, 224, 224, 3)       0         
_________________________________________________________________
block1_conv1 (Conv2D)        (None, 224, 224, 64)      1792      
_________________________________________________________________
block1_conv2 (Conv2D)        (None, 224, 224, 64)      36928     
_________________________________________________________________
block1_pool (MaxPooling2D)   (None, 112, 112, 64)      0         
_________________________________________________________________
block2_conv1 (Conv2D)        (None, 112, 112, 128)     73856     
_________________________________________________________________
block2_conv2 (Conv2D)        (None, 112, 112, 128)     147584    
_________________________________________________________________
block2_pool (MaxPooling2D)   (None, 56, 56, 128)       0         
_________________________________________________________________
block3_conv1 (Conv2D)        (None, 56, 56, 256)       295168    
_________________________________________________________________
block3_conv2 (Conv2D)        (None, 56, 56, 256)       590080    
_________________________________________________________________
block3_conv3 (Conv2D)        (None, 56, 56, 256)       590080    
_________________________________________________________________
block3_pool (MaxPooling2D)   (None, 28, 28, 256)       0         
_________________________________________________________________
block4_conv1 (Conv2D)        (None, 28, 28, 512)       1180160   
_________________________________________________________________
block4_conv2 (Conv2D)        (None, 28, 28, 512)       2359808   
_________________________________________________________________
block4_conv3 (Conv2D)        (None, 28, 28, 512)       2359808   
_________________________________________________________________
block4_pool (MaxPooling2D)   (None, 14, 14, 512)       0         
_________________________________________________________________
block5_conv1 (Conv2D)        (None, 14, 14, 512)       2359808   
_________________________________________________________________
block5_conv2 (Conv2D)        (None, 14, 14, 512)       2359808   
_________________________________________________________________
block5_conv3 (Conv2D)        (None, 14, 14, 512)       2359808   
_________________________________________________________________
block5_pool (MaxPooling2D)   (None, 7, 7, 512)         0         
_________________________________________________________________
flatten (Flatten)            (None, 25088)             0         
_________________________________________________________________
fc1 (Dense)                  (None, 4096)              102764544 
_________________________________________________________________
fc2 (Dense)                  (None, 4096)              16781312  
_________________________________________________________________
predictions (Dense)          (None, 1000)              4097000   
=================================================================

現在のコードでは上記modelのflatten層25088次元の特徴量を抽出していますが、全結合層（fc1）の4096次元の特徴量を抽出するにはどの部分を書き換えればいいか分からず詰まっています。

該当のソースコード

現在のコードは以下の通りです。
今の時点ではinclude_top=Falseとなっているため、feat=model.prediict(x)でflatten層の特徴量が抽出されています。
include_top=Trueにした後、fc1を指定して出力させる方法を教えていただきたいです。

from keras.applications.vgg16 import VGG16, preprocess_input, decode_predictions
from keras.layers import Dense, Dropout, Activation, Flatten
from keras.preprocessing import image

from sklearn.cluster import KMeans
import numpy as np
import pandas as pd
import sys
import cv2
import os
import shutil

DATA_DIR = './'
TARGET_IMAGES_DIR = './'
CLUSTERED_IMAGES_DIR = './'
IMAGE_LABEL_FILE = '.csv'

class Image_Clustering:
	def __init__ (self, n_clusters=3, video_file=None, image_file_temp='img.png', input_video=False):
		self.n_clusters = n_clusters
		self.video_file = video_file
		self.image_file_temp = image_file_temp
		self.input_video = input_video

	def main(self):
		if self.input_video == True:
			self.video_2_frames()
		self.label_images()
		self.classify_images()

	def label_images(self):
		print('Label images...')

		model = VGG16(weights='imagenet', include_top=False)

		images = [f for f in os.listdir(TARGET_IMAGES_DIR) if f[-4:] in ['.jpg']]
		assert(len(images)>0)

		X = []
		for i in range(len(images)):
			feat = self.__feature_extraction(model, TARGET_IMAGES_DIR + images[i])
			X.append(feat)

		X = np.array(X)
		np.save('features.npy',X)
		kmeans = KMeans(n_clusters = self.n_clusters, random_state=0).fit(X)
		print('')
		print('labels:')
		print(kmeans.labels_)
		print('')

		df = pd.DataFrame({'image': images, 'label': kmeans.labels_})
		df.to_csv(DATA_DIR+IMAGE_LABEL_FILE, index=False)

	def __feature_extraction(self, model, img_path):
		img = image.load_img(img_path, target_size=(224,224))
		x = image.img_to_array(img)
		x = np.expand_dims(x, axis=0)
		x = preprocess_input(x)

		feat = model.predict(x)
		feat = feat.flatten()
   
		return feat

		

	def classify_images(self):
		print('Cllassify images...')

		df = pd.read_csv(DATA_DIR+IMAGE_LABEL_FILE)
		labels = list(set(df['label'].values))

		if os.path.exists(CLUSTERED_IMAGES_DIR):
			shutil.rmtree(CLUSTERED_IMAGES_DIR)

		for label in labels:
			print('Copy and paste label %s images.' % label)

			new_dir = CLUSTERED_IMAGES_DIR + str(label) +'/'
			if not os.path.exists(new_dir):
				os.makedirs(new_dir)

			clustered_images = df[df['label']==label]['image'].values
			for ci in clustered_images:
				src = TARGET_IMAGES_DIR + ci
				dst = CLUSTERED_IMAGES_DIR + str(label) + '/' +ci
				shutil.copyfile(src, dst)

if __name__ == "__main__":
	Image_Clustering().main()

補足情報（FW/ツールのバージョンなど）

python2.7
keras2.2.4

Answer 1

イメージとしては、 「VGG modelと Inputは同じで、Outputはfc1」というModelを作れば良いと思います。

from keras.models import Model

...

model = VGG16(weights='imagenet', include_top=True)
new_model = Model(inputs=model.inputs, outputs=model.get_layer('fc1'))
fc1_feat = new_model.predict(x)

みたいな感じでどうでしょうか。（実際に動作は確認せず書いてますが）。

---

■ 追記
あ、すみません、以下が正しいかもしれません。

new_model = Model(inputs=model.inputs, outputs=model.get_layer('fc1').output)

Comments

[sswaka]

追記の通りにModelを設定したら無事に求めたかった特徴量が抽出できました！
ありがとうございます！