What makes a book Non-Fiction?

I think that for a book to be considered Non-Fiction, it should be mostly true. At least 90% of it should be factual and true. There are a few exceptions, though. I believe that if an author is writing something that is "historical non-fiction" or something along those lines, then they shouldn't be allowed to add anything. If you're portraying someone else's life, then you should have to stay 100% true to the facts, because anything you might add could be seen as slanderous. If it's a non-fiction book about someone like Lincoln who has passed away, then you shouldn't be able to add anything to it that isn't proven fact, because there is no way for him to defend himself or call you out for lying. If it's a textbook or something like that, then you should also not be able to add anything that isn't proven to that, because people will use that to back up things that they are saying. If you put something that isn't true into something that people will reference, then it shouldn't be considered non-fiction. If anything it should have to say that it contains your own opinions in it.
However, if you are writing a personal memoir, then who cares if you add information that isn't completely true? If it makes it more interesting to read, and it still stays mainly true, then I see no problem with embellishing a little.