According to the help center:
When a single user continually votes (up or down) on many of your posts within a short period of time, the system considers these votes to be invalid and removes them. This could happen for a variety of reasons, such as a user finding a user's great answer and visiting all of their posts to upvote them, or a user getting into ...
One answer was downvoted by 4 separate individuals, so this could not have been part of the serial downvoting. You had 6 reputation returned to you for 3 out of 5 of the other answers which were serial downvoted, but I flagged for review to see about the other -4 rep. It could be argued that the serial downvoter was appropriate on one answer which means you ...
No, down votes are extremely important here.
We can delete answers, but only for a few reasons:
Doesn't answer the actual question that was asked
Provides no evidence for its claims
Down votes are the main tool you have to say that an answer which addresses the question and provides some evidence or rationale for its conclusions is still ...