The rivets on the fenders are one time use, they must be broken or drilled to be removed unlike the push and pull type plastic pins found elsewhere on the body. More than likely they fell off into the fender wells or bumper while being removed. I wouldn't sweat it

 

You can get a plastic rivet kit which includes plastic rivets that will work on the JL from harbor freight, it's probably fine to leave them out, but it would and has bugged me. I would replace them. I know Amazon sells the plastic rivets also, but not sure about the plastic rivet install tool.