Deleting a budget or the entire group is very easy. A couple of notes:
deleting a budget does not delete transactions visible inside,
deleting a budget group does not delete the budgets that are stored inside -- they'll simply be placed outside the group.
To delete a budget or budget group, please first switch to the Budgets section of MoneyWiz by clicking on the coins button in the menu. Then:
swipe left on the budget (group) name in the sidebar and select "Delete",
enter budget list edit mode by clicking on the Edit button at the bottom of the budgets list, click on the (-) delete button that appears on the left and confirm by clicking on the "Delete" on the right,
bring up the contextual menu and select Delete.