Definitely visit the castle district. Also, you may want to experience one of the many thermal baths - very soothing and relaxing and a very Hungarian thing to do.
Here's the Budapest Tourism webpage with loads more ideas:
http://www.budapestinfo.hu/en/
And as an FYI, if you are taking the subway there and you see the doors starting to close do NOT try to squeeze in - the door slam closed violently. They show up often enough anyway, so not worth injuries to try and catch one that's about to leave.