One of the less intuitive things in Unreal World Outliner is the the way how to hide lights or other objects. Disabling them by clicking on the eye in World Outliner results them turning off in the viewport, however reloading a scene will re-enable them causing confusion. Same goes for everything else.

Better way For lights, toggle Affects World to disable/enable it without deleting the actor/component.

For meshes, toggle either Visible (completely hides it) and/or Hidden in Game (hides in game view but not in editor view) to hide it.

This is definitely a head-scratcher. I think better way is to not to use the eye icon at all.