Automating file input

[qary busey]

Hello.

I am glad to find a place like this.

I wrote a program that looks at 9 different
economic and financial statistics.

I manually update the 9 files once a week.

The program prints file names with the files$
command and then prompts for a single file.
I type in the file I want to use and it is loaded
into an array.I use the program to make moving
averages and charts.

I want to automate the program. I have located
a data provider that can give me hundreds of
individual stock files and other data which
are automatically updated at the end of the day.

I want the program to automatically load in one
file, do some data manipulation on it, produce a
report and then go on to the next file.

My problem is how do I automate going from
one file to the next without actually specifying
each file.
The files are all in one directory with the program.

Thanks

[burger2227]

Number them.

Code: Select all

FOR num% = 1 TO 9
filename$ = "Data" + LTRIM$(STR$(num%)) + ".DAT"
OPEN filename$ FOR INPUT AS #1

'do calculations

CLOSE #1
NEXT

BTW, QB64 can compile QB code and perhaps retrieve the data from the site using TCP/IP functionality too! The link is in my signature.

Ted

[qary busey]

I will put this in.

Thanks

[qary busey]

hi


I may not be seeing it, but is there a way to use this code to open and close hundreds of data files automatically?

I have used READ and DATA in the past for a small number of files but I would like to change to working with hundreds or even thousands of files opened and closed automatically. The number would vary by a few each day
because the stock exchange delists a few stocks and adds a few each week.

I won't be able to respond till later this evening.

thanks

[burger2227]

Well you have up to 99999999.DAT or 100 million using a DOUBLE number #.

[qary busey]

thanks for the answer.

[Theunis]

Many years ago I wrote a small program to read the directory and then played the processed music files. Too long ago and too lost that program be.

If the files normally have the same extension say *.txt then You could write a program to read the directory of interest, Something like this
SHELL "DIR C:\QB45\*.TXT > MYDIR.TXT "

Using a word processor would be easy but if you don't have time to edit non relevant info out then your program will then have to open MYDIR.TXT and Read the file names (process the file names - the first 8 characters + "." + the next 3 characters are of interest) into an array, From where you can process them one by one. Using a FOR.... NEXT loop.

If only two or three extensions then the *.TXT and MYDIR.TXT can be changed to the other EXT and you will have have to insert "SHELL ..." for the other extensions.
If every file has its own extension then *.TXT must be changed to *.* and you will have a lot more processing to do to extract the file names.
With WIN XP the *.* file looks something like this:
*************************************

Volume in drive C is D40GIG
Volume Serial Number is 1360-14ED
Directory of C:\QB45

. <DIR> 10-07-26 16:54
.. <DIR> 10-07-26 16:54
EGAVGA~1 FON 3570 01-04-09 9:50
SHARLE~1 <DIR> 10-11-29 11:06
DRAWCHAR BAS 4499 09-03-23 22:52
GALEST~1 WAV 25785060 10-06-26 17:37
398 file(s) 36035367 bytes
1023932928 bytes free
*************************************
Which you will have to process to extract the file names.
If you download files daily then do it to a specific directory and process that directory. After they have been processed move them to another directory.

[burger2227]

DIR option /B only lists the file names and eliminates the directory info.

SHELL "DIR C:\QB45\*.TXT /B > MYDIR.TXT "

In fact, you can verify that a file exists by putting the entire DOS 8.3 file name in the SHELL.

Code: Select all

SHELL "DIR C:\QB45\TestFile.TXT /b > MYDIR.TXT"
OPEN "MYDIR.TXT" FOR APPEND AS #1
L = LOF(1)
CLOSE #1
KILL "MYDIR.TXT"  'you were gonna delete it anyhow
IF L THEN PRINT "File exists!" ELSE PRINT "File not found"

[Theunis]

@ Ted
Now that is much easier.
I haven't used the DIR command for ages.
The last time I tried it was on WIN98SE, MS has since enhanced the DIR command.
I tried /b on the DOS command line and it gave me the long file names.
I see doing it from QB4.5 gives the DOS 8.3 names.

It should now be pudding for him to do what he wants.

[burger2227]

QB64 can use long file names. It needs

SHELL _HIDE "DIR C:\QB45\TestFile.TXT /b > MYDIR.TXT"

to send data to a file.

[qary busey]

I found a program called CdList which must do the the same things you are talking about.

It lists all the files in a directory and has a four line header. It can add numbering in front if you want.

All the data files I want to work with have the same extention. I wrote a program that reads the list of files created by CdList .
The program dumps the header and starts reading the files one by one. After it opens the first file, the data is loaded into an array and sent for calculations. If the calculations meet certain criteria then the result is printed to a file. then the program goes back and reads the next file etc.

[burger2227]

Here is a copy of the DIR$ function that is used in PDS(7.1). Enter *.BAS to get a list of the BAS files or any other kind. After a spec is entered you can request another file on the list by using an empty string "". For filenames with a space in DOS use quotes around the name. LINE INPUT allows quotes. This could all be automated to fit your program needs.

The temporary file is deleted so make sure that you use a name that doesn't already exist. DIR$INF0 uses a zero instead of an O to prevent that.

Code: Select all

  
FOR i = 1 TO 2
  LINE INPUT "Enter a file spec: ", spec$
  file$ = DIR$(spec$)        'use a file spec ONCE to find the last file name listed
  PRINT DIRCount%, file$,    'function can return the file count using SHARED variable 
  DO
    K$ = INPUT$(1)
    file$ = DIR$("")         'use an empty string parameter to return a list of files!
    PRINT file$,
  LOOP UNTIL LEN(file$) = 0  'file list ends with an empty string
NEXT
END

FUNCTION DIR$ (spec$)
CONST TmpFile$ = "DIR$INF0.INF", ListMAX% = 500  'change maximum to suit your needs
SHARED DIRCount%                                 'returns file count if desired
STATIC Ready%, Index%, DirList$()
IF NOT Ready% THEN REDIM DirList$(ListMax%): Ready% = -1  'DIM array first use
IF spec$ > "" THEN                               'get file names when a spec is given
  SHELL _HIDE "DIR " + spec$ + " /b > " + TmpFile$
  Index% = 0: DirList$(Index%) = "": ff% = FREEFILE
  OPEN TmpFile$ FOR APPEND AS #ff%
  size& = LOF(ff%) 
  CLOSE #ff%
  IF size& = 0 THEN KILL TmpFile$: EXIT FUNCTION
  OPEN TmpFile$ FOR INPUT AS #ff%     
  DO WHILE NOT EOF(ff%) AND Index% < ListMAX%
    Index% = Index% + 1
    LINE INPUT #ff%, DirList$(Index%)   
  LOOP
  DIRCount% = Index%                       'SHARED variable can return the file count
  CLOSE #ff%
  KILL TmpFile$
ELSE IF Index% > 0 THEN Index% = Index% - 1 'no spec sends next file name
END IF                                      
DIR$ = DirList$(Index%)
END FUNCTION  

One thing extra is that the number of files, DIRCount%, can be referenced in the main program if needed after the spec call. If no file exists, it returns 0 and an empty string. An empty string is returned after the last file in list too.

Ted

[Theunis]

A bit late. I scratched and scratched until I found something. I turns out I did not use the Dir command. I used LOOK.EXE which is brilliant for this purpose- abolutely no processing is required. I tried it in XP and it still gives 8.3 DOS names. Using /Q switch it gives Drive\Path\Filename (Nothing else). If anyone is interested I still have LOOK.EXE plus it's DOC file and can make a ZIP of it and post it on DROPBOX.