See the above figure ( from What is the difference between a Read and a Fragment in RNA-seq? )
I guess that depends on the person who is using it.
I would say (for Illumina): Fragment size is the size of the actual fragments that you put into the sequencer (including adapter sequences). Insert size is the length of the DNA (or RNA) that you want to sequence and that is "inserted" between the adapters (so adapters excluded).
BUT fragment size yould also be the size of the fragments after shearing which would be insert size by my definition above.