It's interesting that he says that he was afraid - it is his master who accuses him of being wicked and lazy.
And so I guess the question comes as to what he was afraid of - and from his explanation it is about who his master is and how he behaved. Maybe he didn't want to be associated with this type of activity and was afraid of that - maybe thinking of would have to live with the people around him if his master didn't return.
If that's the case its not intended to make us think of Jesus in the same was as the master in the servant's story but it is to make us think about whether we are prepared to be seen to be associated with him - with whatever gifts and abilities he has given us.