Impact Factors are hugely over-rated and it is widely acknowledged that they are a deeply flawed concept. They simply tell you what the average impact of other papers published in that journal is. That does not affect the quality or impact of your paper whatsoever.
In my opinion the only thing you should really care about is in publishing fully open-access. That will give you more visibility and potentially more impact if the community appreciate your paper.