I am starting to work on my scenery and wanted to use the real thing for dirt in some corrals. Turns out my dirt is magnetic. Seems to me I read somewhere that is not a good thing. Am I asking for trouble if I use it ? Was planning on using white glue with it.