According to the help center:
When a single user continually votes (up or down) on many of your posts within a short period of time, the system considers these votes to be invalid and removes them. ...
No, down votes are extremely important here.
We can delete answers, but only for a few reasons:
Doesn't answer the actual question that was asked
Provides no evidence for its claims