I use as little custom bricks as possible and I've never loving ever used a brick from a filler brick pack.
If you have to use filler bricks when building, you aren't trying hard enough.
(though there are some bricks that I would like to be a part of the default brickset)