2004 dodge dakota sport ext cab v-6 2wd

when I turn the key to start all the lights come on as normal but you have to sit with key in start position until it wants to start  about 2-3 min  or you can turn key on and sit for a few min 3-5  then try starting and usually will start or just sit with key turned and wait some more.  whats happening I was thinking the ignition relay switch but thats a guess