Is olive oil or vaseline bad for you penus if you use it as a lubricant?

Not at all. It's totally safe to use. However, you may want to wash it off when you finished.