How to compute shared memory anchor address in VxWorks 6.3 - embedded

Question

I've inherited 3 SBS (now GE Fanuc) C2k single board computers connected over a CompactPCI backplane, and need to get shared memory up and running between them. My understanding is that in previous years, the value for SM_ANCHOR_ADRS was initially "found" by someone, and whenever PMCs were swapped in or out the value for SM_ANCHOR_ADRS was fiddled with until it worked again. This time around, instead of randomly poking memory addresses, I'd like to learn how this value is come about to begin with.
I have noticed in the VxMP manual there's a description of how to compute this address for VME, given a memory map and known offsets. I've tried looking a similar memory map in the C2k user's manual and none exists. (The bits about PCI auto-configuration I saw in the board support package may have something to do with that?)
Also, in case it's useful, relevant bits from the kernel configuration:
Master:
SM_MEM_ADRS NONE
SM_ANCHOR_ADRS (char*)0x4100
SM_ANCHOR_OFFSET 0x4100
SM_INT_TYPE SM_INT_NONE
SM_MAX_WAIT 3000
Slave cards:
SM_MEM_ADRS (SM_ANCHOR_ADRS)
SM_ANCHOR_ADRS (char*)0x84004100 //this is the number that generally gets fiddled with..
SM_ANCHOR_OFFSET 0x4100
SM_INT_TYPE SM_INT_NONE
SM_MAX_WAIT 3000
With this info, how would I go about computing the shared memory anchor address for VxWorks for devices on a CompactPCI backplane?

Answer 1

This is really a function of the PCI bus architecture and its dynamic nature.
The SM_ANCHOR_ADRS represents the shared memory in the PCI address space of the slave card.
The issue is that when you add more cards, the PCI address space can potentially change.
If you use the PCI Auto Configuration you essentially let vxWorks assign the addresses for you. I'm not sure of the assignment order, but I think the address space is allocated in the order devices are detected on the PCI bus.
Let's see if I can illustrate:
PCI Address 3 Devices 4 Devices (Added 1)
0x80000000 (base) +------------------+ +---------------------+
| Host Bridge | | Host Bridge |
| Device (0,0,0) | | Device (0,0,0) |
| 0x01000000 size | | 0x01000000 size |
0x81000000 +------------------+ +---------------------+
| Slave 1 | | Slave 1 |
| Device (0,4,0) | | Device (0,4,0) |
| 0x01000000 size | | 0x01000000 size |
0x82000000 +------------------+ +---------------------+
| Slave 2 | | CPCI Card |
| Device (0,8,0) | | Device (0,6,0) |
| 0x01000000 size | | 0x01000000 size |
0x83000000 +------------------+ +---------------------+
| Slave 2 |
| Device (0,8,0) |
| 0x01000000 size |
+---------------------+
The Device tuple represents (Bus #, Device #, Function #).
The Device # is determined by which slot the PCI card is plugged in. Throw in the possibility of multiple PCI buses and the order in which things get processed can get complicated.
When using the PCI Auto configuration, you specify the base address and size of the PCI memory windows (there are multiple possible PCI memory windows).
The code then goes on the PCI bus and detects the devices. As a device is detected, it tells the PCI system how much memory it needs and the system maps the addresses within the PCI window for that device. In my example, all devices requested 0x01000000 of RAM.
This works well when the PCI configuration is static and doesn't change.
However, as we can see, if we add a new device (the CPCI), then the order of the devices changed, which causes some of the addresses to also change.
PCI Autoconfiguration simplifies the life since you don't have to worry about configuring all the devices ahead of time. They are just discovered.
However, if the configuration is variable, you probably need to configure things manually: i.e. know ahead of time what devices might be present and attempt to configure them. One the upside, you have full control over what address you assign to a given PCI device.
To get back to your question, how do you know the address? If you have access to the vxWorks shell, you can use the pciConfigTopoShow(), which should show you the various PCI devices found and their address spaces.