If the copier has colour scan type available make sure your scan settings are defaulted to B/W text. If it is defaulted to colour file size will increase.
Also when you say it is a text document. I assume this means there is no shading behind the text. I had a customer complaining about file size and when I arrived on site I find they are scanning 50page docs with background shading all over them.
