Although they're weak, they have strength in numbers. Don't engage.
Avoid them, because if they board the ship, you'll be in a lot more trouble.
Your options are:
1) Attack them at a distance and destroy them before they board; risk provoking reinforcements.
2) Try to outmaneuver them, and jump when available; risk them boarding you in the process.
In other words. Do NOT let them board.