将 AXI 设计拆分到两片FPGA

将 AXI 设计拆分到两片FPGA一、内容介绍二、基础知识介绍2.1 Veloce Prototyping System (VPS) 与 proFPGA2.2 Xilinx XCVU19P FPGA2.3 AXI 总线与 XDMA IP2.4 设计分割原理2.5 软件工具链三、操作步骤3.1 设置环境变量并启动授权服务3.2 创建工程文件3.2.1 `pcie_and_connector_bd.tcl` —— Vivado Block Design 脚本3.2.2 `build_pcie_and_connector_stub.tcl` —— OOC 综合脚本3.2.3 `simple_ctrl.v` —— 用户 AXI-Lite 外设3.2.4 `system.v` —— 顶层集成文件3.2.5 `vortex.cfg` —— proFPGA 系统配置文件3.2.6 `pcie.xdc` —— 物理与分区约束3.2.7 `redrun.tcl`、`redcomp.tcl` 等编译控制文件3.3 生成 bitstream3.4 查看 PCIe 拓扑3.5 重启 proFPGA 系统3.6 下载 bitstream3.7 PCIe 热插拔与 BAR 空间确认3.8 编写并加载 Linux 驱动，完成功能测试3.8.1 驱动代码 `lite_pcie_drv.c`3.8.2 编译并加载驱动3.9 退出 redrun 并关闭 proFPGA一、内容介绍在复杂的 ASIC/SoC 原型验证中，单片 FPGA 的容量往往无法容纳整个设计。本指南演示如何利用Veloce Prototyping System (VPS)（基于 ProDesign proFPGA 硬件平台）将一个包含AXI 总线与PCI Express 接口的设计，拆分到两片 FPGA 上，并通过板间互联实现协同工作。具体来说，本示例包含：一块 FPGA (FB1)：负责实现 PCIe Gen3 x8 接口、XDMA IP 核及 AXI 互联。另一块 FPGA (FA1)：实现一个简单的 AXI-Lite 控制模块 (simple_ctrl)，该模块能通过配置寄存器发起一次 AXI 写事务，经 PCIe 将数据写入主机内存。整个流程覆盖了从 Vivado Block Design 构建、OOC 综合、多 FPGA 设计分割与编译、bitstream 下载，到 Linux PCIe 驱动验证的完整步骤，可作为复杂设计跨 FPGA 分割的入门参考。文档中的命令均基于实际工程，您可以直接复制执行。二、基础知识介绍在开始操作之前，了解以下核心概念有助于理解每一步的意图。2.1 Veloce Prototyping System (VPS) 与 proFPGAVeloce Prototyping System是西门子 EDA 提供的高性能企业级原型验证平台。本指南使用的硬件为 ProDesign 公司的proFPGA 多 FPGA 系统，它常作为 VPS 的硬件载体。proFPGA 采用模块化主板，可插接多个 FPGA 子板，并通过专用的高速线缆 (ICC) 在 FPGA 之间传递信号。2.2 Xilinx XCVU19P FPGAXCVU19P 属于 Virtex UltraScale+ 系列，拥有大量逻辑单元、高速收发器 (GTY)，支持 PCIe Gen3/Gen4 硬核，是原型验证的理想器件。2.3 AXI 总线与 XDMA IPAXI (Advanced eXtensible Interface)：ARM 推出的片上总线协议，广泛用于 FPGA 设计。本示例使用了 AXI4-Lite（轻量级）和 AXI4 全协议。XDMA IP：Xilinx 提供的 DMA/Bridge Subsystem for PCI Express，可在 PCIe 与 AXI 内存映射接口之间进行高性能数据传输。这里配置为 AXI Bridge 模式，将 PCIe BAR 空间映射到 AXI 地址空间。2.4 设计分割原理将一个大的数字系统划分到多片 FPGA 上，需要：逻辑分区：决定哪些模块放在哪片 FPGA。本例中将 PCIe/XDMA 置于 FB1，用户逻辑simple_ctrl置于 FA1。信号互连：通过 proFPGA 的板间线缆 (ICC) 将两片 FPGA 的 I/O 连接起来，传递跨分区的 AXI 信号。分割工具：ProDesign 的red工具链 (redlib,redmap,redcomp) 可以自动处理多 FPGA 设计的综合、分割、布局布线及 bitstream 生成。我们通过red.force文件告诉工具哪些信号需要跨 FPGA 传输，pcie.xdc则定义物理引脚约束和分区位置。2.5 软件工具链Vivado 2020.2：用于创建 Block Design、综合 XDMA 子系统并生成 DCP（Design Checkpoint）和 stub 文件。proFPGA Builder / red 套件：用于板级配置生成、多 FPGA 编译、调试与下载。Linux 驱动：简单的内核模块，通过 PCIe BAR 访问 FPGA 寄存器，验证数据写入主机内存。三、操作步骤注意：以下命令均在 Linux 环境下执行，假定已正确安装 Vivado 2020.2、proFPGA 软件套件及相关 license。3.1 设置环境变量并启动授权服务首先进入 proFPGA 安装目录，加载环境脚本并启动 license 服务。cd/opt/sourceenv.sh ./start_license_server.sh3.2 创建工程文件建立一个工作目录，所有设计文件都将放在此处。mkdir-p/home/vps_lite_democd/home/vps_lite_demo下面我们将逐一创建必需的文件。每个文件都通过cat file 'EOF' ... EOF命令直接写入，方便您复制。3.2.1pcie_and_connector_bd.tcl—— Vivado Block Design 脚本该 Tcl 脚本用于在 Vivado 中自动构建一个pcie_and_connector_stub设计，包含：XDMA IP(PCIe Gen3 x8, AXI Bridge 模式)AXI SmartConnect / Interconnect用于路由 AXI 总线差分时钟缓冲器(IBUFDSGTE) 为 PCIe 提供参考时钟对外露出 AXI4-Lite 主/从接口，以便与外部用户逻辑连接cat pcie_and_connector_bd.tcl 'EOF' ################################################################ # This is a generated script based on design: pcie_and_connector_stub # # Though there are limitations about the generated script, # the main purpose of this utility is to make learning # IP Integrator Tcl commands easier. ################################################################ namespace eval _tcl { proc get_script_folder {} { set script_path [file normalize [info script]] set script_folder [file dirname $script_path] return $script_folder } } variable script_folder set script_folder [_tcl::get_script_folder] ################################################################ # Check if script is running in correct Vivado version. ################################################################ set scripts_vivado_version 2020.2 set current_vivado_version [version -short] if { [string first $scripts_vivado_version $current_vivado_version] == -1 } { puts "" catch {common::send_gid_msg -ssname BD::TCL -id 2041 -severity "ERROR" "This script was generated using Vivado $scripts_vivado_version and is being run in $current_vivado_version of Vivado. Please run the script in Vivado $scripts_vivado_version then open the design in Vivado $current_vivado_version. Upgrade the design by running \"Tools = Report = Report IP Status...\", then run write_bd_tcl to create an updated script."} return 1 } ################################################################ # START ################################################################ # To test this script, run the following commands from Vivado Tcl console: # source pcie_and_connector_stub_script.tcl # If there is no project opened, this script will create a # project, but make sure you do not have an existing project # ./myproj/project_1.xpr in the current working folder. set list_projs [get_projects -quiet] if { $list_projs eq "" } { create_project project_1 myproj -part xcvu19p-fsva3824-2-e } # CHANGE DESIGN NAME HERE variable design_name set design_name pcie_and_connector_stub # If you do not already have an existing IP Integrator design open, # you can create a design using the following command: # create_bd_design $design_name # Creating design if needed set errMsg "" set nRet 0 set cur_design [current_bd_design -quiet] set list_cells [get_bd_cells -quiet] if { ${design_name} eq "" } { # USE CASES: # 1) Design_name not set set errMsg "Please set the variable design_name to a non-empty value." set nRet 1 } elseif { ${cur_design} ne "" ${list_cells} eq "" } { # USE CASES: # 2): Current design opened AND is empty AND names same. # 3): Current design opened AND is empty AND names diff; design_name NOT in project. # 4): Current design opened AND is empty AND names diff; design_name exists in project. if { $cur_design ne $design_name } { common::send_gid_msg -ssname BD::TCL -id 2001 -severity "INFO" "Changing value of design_name from $design_name to $cur_design since current design is empty." set design_name [get_property NAME $cur_design] } common::send_gid_msg -ssname BD::TCL -id 2002 -severity "INFO" "Constructing design in IPI design $cur_design..." } elseif { ${cur_design} ne "" $list_cells ne "" $cur_design eq $design_name } { # USE CASES: # 5) Current design opened AND has components AND same names. set errMsg "Design $design_name already exists in your project, please set the variable design_name to another value." set nRet 1 } elseif { [get_files -quiet ${design_name}.bd] ne "" } { # USE CASES: # 6) Current opened design, has components, but diff names, design_name exists in project. # 7) No opened design, design_name exists in project. set errMsg "Design $design_name already exists in your project, please set the variable design_name to another value." set nRet 2 } else { # USE CASES: # 8) No opened design, design_name not in project. # 9) Current opened design, has components, but diff names, design_name not in project. common::send_gid_msg -ssname BD::TCL -id 2003 -severity "INFO" "Currently there is no design $design_name in project, so creating one..." create_bd_design $design_name common::send_gid_msg -ssname BD::TCL -id 2004 -severity "INFO" "Making design $design_name as current_bd_design." current_bd_design $design_name } common::send_gid_msg -ssname BD::TCL -id 2005 -severity "INFO" "Currently the variable design_name is equal to \"$design_name\"." if { $nRet != 0 } { catch {common::send_gid_msg -ssname BD::TCL -id 2006 -severity "ERROR" $errMsg} return $nRet } set bCheckIPsPassed 1 ################################################################## # CHECK IPs ################################################################## set bCheckIPs 1 if { $bCheckIPs == 1 } { set list_check_ips "\ xilinx.com:ip:smartconnect:1.0\ xilinx.com:ip:util_ds_buf:2.1\ xilinx.com:ip:xdma:4.1\ " set list_ips_missing "" common::send_gid_msg -ssname BD::TCL -id 2011 -severity "INFO" "Checking if the following IPs exist in the project's IP catalog: $list_check_ips ." foreach ip_vlnv $list_check_ips { set ip_obj [get_ipdefs -all $ip_vlnv] if { $ip_obj eq "" } { lappend list_ips_missing $ip_vlnv } } if { $list_ips_missing ne "" } { catch {common::send_gid_msg -ssname BD::TCL -id 2012 -severity "ERROR" "The following IPs are not found in the IP Catalog:\n $list_ips_missing\n\nResolution: Please add the repository containing the IP(s) to the project." } set bCheckIPsPassed 0 } } if { $bCheckIPsPassed != 1 } { common::send_gid_msg -ssname BD::TCL -id 2023 -severity "WARNING" "Will not continue with creation of design due to the error(s) above." return 3 } ################################################################## # DESIGN PROCs ################################################################## # Procedure to create entire design; Provide argument to make # procedure reusable. If parentCell is "", will use root. proc create_root_design { parentCell } { variable script_folder variable design_name if { $parentCell eq "" } { set parentCell [get_bd_cells /] } # Get object for parentCell set parentObj [get_bd_cells $parentCell] if { $parentObj == "" } { catch {common::send_gid_msg -ssname BD::TCL -id 2090 -severity "ERROR" "Unable to find parent cell $parentCell!"} return } # Make sure parentObj is hier blk set parentType [get_property TYPE $parentObj] if { $parentType ne "hier" } { catch {common::send_gid_msg -ssname BD::TCL -id 2091 -severity "ERROR" "Parent $parentObj has TYPE = $parentType. Expected to be hier."} return } # Save current instance; Restore later set oldCurInst [current_bd_instance .] # Set parent object as current current_bd_instance $parentObj # Create interface ports set M01_AXI_0 [ create_bd_intf_port -mode Master -vlnv xilinx.com:interface:aximm_rtl:1.0 M01_AXI_0 ] set_property -dict [ list \ CONFIG.ADDR_WIDTH {32} \ CONFIG.CLK_DOMAIN {pcie_and_connector_stub_CLK} \ CONFIG.DATA_WIDTH {32} \ CONFIG.FREQ_HZ {25000000} \ CONFIG.NUM_READ_OUTSTANDING {1} \ CONFIG.NUM_WRITE_OUTSTANDING {1} \ CONFIG.PROTOCOL {AXI4LITE} \ CONFIG.READ_WRITE_MODE {READ_WRITE} \ ] $M01_AXI_0 set S00_AXI_0 [ create_bd_intf_port -mode Slave -vlnv xilinx.com:interface:aximm_rtl:1.0 S00_AXI_0 ] set_property -dict [ list \ CONFIG.ADDR_WIDTH {32} \ CONFIG.ARUSER_WIDTH {0} \ CONFIG.AWUSER_WIDTH {0} \ CONFIG.BUSER_WIDTH {0} \ CONFIG.DATA_WIDTH {32} \ CONFIG.FREQ_HZ {25000000} \ CONFIG.HAS_BRESP {1} \ CONFIG.HAS_BURST {1} \ CONFIG.HAS_CACHE {1} \ CONFIG.HAS_LOCK {1} \ CONFIG.HAS_PROT {1} \ CONFIG.HAS_QOS {1} \ CONFIG.HAS_REGION {1} \ CONFIG.HAS_RRESP {1} \ CONFIG.HAS_WSTRB {1} \ CONFIG.ID_WIDTH {0} \ CONFIG.MAX_BURST_LENGTH {1} \ CONFIG.NUM_READ_OUTSTANDING {1} \ CONFIG.NUM_READ_THREADS {1} \ CONFIG.NUM_WRITE_OUTSTANDING {1} \ CONFIG.NUM_WRITE_THREADS {1} \ CONFIG.PROTOCOL {AXI4LITE} \ CONFIG.READ_WRITE_MODE {WRITE_ONLY} \ CONFIG.RUSER_BITS_PER_BYTE {0} \ CONFIG.RUSER_WIDTH {0} \ CONFIG.SUPPORTS_NARROW_BURST {0} \ CONFIG.WUSER_BITS_PER_BYTE {0} \ CONFIG.WUSER_WIDTH {0} \ ] $S00_AXI_0 set diff_clock_rtl_0 [ create_bd_intf_port -mode Slave -vlnv xilinx.com:interface:diff_clock_rtl:1.0 diff_clock_rtl_0 ] set_property -dict [ list \ CONFIG.FREQ_HZ {100000000} \ ] $diff_clock_rtl_0 set pcie_7x_mgt_rtl_0 [ create_bd_intf_port -mode Master -vlnv xilinx.com:interface:pcie_7x_mgt_rtl:1.0 pcie_7x_mgt_rtl_0 ] # Create ports set CLK [ create_bd_port -dir I -type clk -freq_hz 25000000 CLK ] set_property -dict [ list \ CONFIG.ASSOCIATED_ASYNC_RESET {} \ CONFIG.ASSOCIATED_BUSIF {S00_AXI_0:M01_AXI_0} \ CONFIG.ASSOCIATED_CLKEN {} \ CONFIG.ASSOCIATED_RESET {} \ ] $CLK set RESETN [ create_bd_port -dir I -type rst RESETN ] set_property -dict [ list \ CONFIG.POLARITY {ACTIVE_LOW} \ ] $RESETN set axi_aresetn [ create_bd_port -dir O -type rst axi_aresetn ] set reset_rtl_0 [ create_bd_port -dir I -type rst reset_rtl_0 ] set_property -dict [ list \ CONFIG.POLARITY {ACTIVE_LOW} \ ] $reset_rtl_0 # Create instance: axi_interconnect_0, and set properties set axi_interconnect_0 [ create_bd_cell -type ip -vlnv xilinx.com:ip:axi_interconnect:2.1 axi_interconnect_0 ] set_property -dict [ list \ CONFIG.NUM_MI {1} \ ] $axi_interconnect_0 # Create instance: smartconnect_0, and set properties set smartconnect_0 [ create_bd_cell -type ip -vlnv xilinx.com:ip:smartconnect:1.0 smartconnect_0 ] set_property -dict [ list \ CONFIG.HAS_ARESETN {0} \ CONFIG.NUM_CLKS {2} \ CONFIG.NUM_SI {1} \ ] $smartconnect_0 # Create instance: smartconnect_1, and set properties set smartconnect_1 [ create_bd_cell -type ip -vlnv xilinx.com:ip:smartconnect:1.0 smartconnect_1 ] set_property -dict [ list \ CONFIG.HAS_ARESETN {0} \ CONFIG.NUM_CLKS {2} \ CONFIG.NUM_MI {2} \ CONFIG.NUM_SI {1} \ ] $smartconnect_1 # Create instance: util_ds_buf_0, and set properties set util_ds_buf_0 [ create_bd_cell -type ip -vlnv xilinx.com:ip:util_ds_buf:2.1 util_ds_buf_0 ] set_property -dict [ list \ CONFIG.C_BUF_TYPE {IBUFDSGTE} \ ] $util_ds_buf_0 # Create instance: xdma_0, and set properties set xdma_0 [ create_bd_cell -type ip -vlnv xilinx.com:ip:xdma:4.1 xdma_0 ] set_property -dict [ list \ CONFIG.PF0_DEVICE_ID_mqdma {9038} \ CONFIG.PF2_DEVICE_ID_mqdma {9038} \ CONFIG.PF3_DEVICE_ID_mqdma {9038} \ CONFIG.axi_addr_width {32} \ CONFIG.axi_data_width {256_bit} \ CONFIG.axibar2pciebar_0 {0x0000000080000000} \ CONFIG.axibar_num {1} \ CONFIG.axisten_freq {250} \ CONFIG.c_s_axi_supports_narrow_burst {false} \ CONFIG.cfg_ext_if {true} \ CONFIG.coreclk_freq {500} \ CONFIG.disable_gt_loc {true} \ CONFIG.en_gt_selection {true} \ CONFIG.functional_mode {AXI_Bridge} \ CONFIG.mode_selection {Advanced} \ CONFIG.pcie_blk_locn {PCIE4C_X0Y3} \ CONFIG.pciebar2axibar_1 {0x0000000000000000} \ CONFIG.pf0_bar0_enabled {true} \ CONFIG.pf0_bar0_size {512} \ CONFIG.pf0_bar1_enabled {false} \ CONFIG.pf0_bar1_size {4} \ CONFIG.pf0_device_id {9038} \ CONFIG.pl_link_cap_max_link_speed {8.0_GT/s} \ CONFIG.pl_link_cap_max_link_width {X8} \ CONFIG.plltype {QPLL1} \ CONFIG.select_quad {GTY_Quad_227} \ CONFIG.xdma_axilite_slave {true} \ ] $xdma_0 # Create interface connections connect_bd_intf_net -intf_net S00_AXI_0_1 [get_bd_intf_ports S00_AXI_0] [get_bd_intf_pins smartconnect_0/S00_AXI] connect_bd_intf_net -intf_net axi_interconnect_0_M00_AXI [get_bd_intf_pins axi_interconnect_0/M00_AXI] [get_bd_intf_pins smartconnect_1/S00_AXI] connect_bd_intf_net -intf_net diff_clock_rtl_0_1 [get_bd_intf_ports diff_clock_rtl_0] [get_bd_intf_pins util_ds_buf_0/CLK_IN_D] connect_bd_intf_net -intf_net smartconnect_0_M00_AXI [get_bd_intf_pins smartconnect_0/M00_AXI] [get_bd_intf_pins xdma_0/S_AXI_B] connect_bd_intf_net -intf_net smartconnect_1_M00_AXI [get_bd_intf_pins smartconnect_1/M00_AXI] [get_bd_intf_pins xdma_0/S_AXI_LITE] connect_bd_intf_net -intf_net smartconnect_1_M01_AXI [get_bd_intf_ports M01_AXI_0] [get_bd_intf_pins smartconnect_1/M01_AXI] connect_bd_intf_net -intf_n