inbred children are significantly more likely to have birth defects, but the stereotype of subhuman mongoloid families in the mountains of west virginia, for example, is untrue.
otherwise this is really a moral vs ethical issue. cultures generally see it as taboo because we're naturally not as* attracted to close relatives thanks to evolution optimizing us for genetic variety, alongside the direct results that caused evolution to promote variety
pragmatically though there's nothing wrong with a sister and brother marrying and adopting
*i say as, because it's a result of growing up closely to the relative in question, and it's not actually something our heads can recognize as genetic. cousin-loving isn't nearly as universally condemned because of this.