Model Performance is just 88%

[lisn3]

I trained the model using parameters in the paper for 300 epochs, but the final training accuracy is just 88%, and the testing F1-score is 88%. Could you please give some ideas? Has anyone got a higher performance?

[wuyifan18]

@lisn3 try to use list to generate the full dataset, just like this:

DeepLog/LogKeyModel_predict.py

Lines 11 to 22 in 502aaf0

	# If you what to replicate the DeepLog paper results(Actually, I have a better result than DeepLog paper results),
	# you should use the 'list' not 'set' to obtain the full dataset, I use 'set' just for test and acceleration.
	hdfs = set()
	# hdfs = []
	with open('data/' + name, 'r') as f:
	for ln in f.readlines():
	ln = list(map(lambda n: n - 1, map(int, ln.strip().split())))
	ln = ln + [-1] * (window_size + 1 - len(ln))
	hdfs.add(tuple(ln))
	# hdfs.append(tuple(ln))
	print('Number of sessions({}): {}'.format(name, len(hdfs)))
	return hdfs

[lisn3]

Thanks, I will try again.

[Abraham12580]

Thanks, I will try again.

hi，have your performance gotten improved yet?

[ldselvera]

Unfortunately, that will be the highest performance with this implementation. The only way to increase it is by modifying the model and fine tuning parameters.

[hayhan]

Use one-hot vector for input data may help.

[gutjuri]

If you do what wuifan suggested (replacing set with list) you'll get a tremendously increased performance.

elapsed_time: 2877.333s
false positive (FP): 841, false negative (FN): 381, Precision: 95.138%, Recall: 97.737%, F1-measure: 96.420%

[Abraham12580]

您好，我已经收到你的邮件。祝您工作顺利，天天开心思密达~                                                                 唐子超