As I wasn't getting it in my test I stubbornly refused to believe it was me!
After spending some time investigating and coming across the following post:
http://social.microsoft.com/forums/en-US/1b9284f7-4f94-447f-9177-616ecbe640ed/business-process-error-with-activityfeedsplugins
http://social.microsoft.com/forums/en-US/1b9284f7-4f94-447f-9177-616ecbe640ed/business-process-error-with-activityfeedsplugins
I discovered that I had created a plugin step in the Activity Feed plugin!
So if you get a Business Process Error in the ActivityFeeds.Plugins then I suggest you look for some plugin steps in the corresponding item (in my case ActivityClose).