meta data for this page
  •  

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revisionPrevious revision
Next revision		Previous revision
hw:server:10gbe [2022/10/26 21:27] – niziakhw:server:10gbe [2022/10/27 21:12] (current) – niziak
Line 1: Line 1:
-====== Emulex ======+====== Emulex OCe11102 ====== 
 + 
 +2-PORT IBM/EMULEX 10GB VIRTUAL FABRIC ADAPTER 49Y7952 OCE11102 HIGH  
 +  - Low Profile 
 +  - Pci Express 2.0 X8 
 +  - IBM 49Y7952 
 +  - ~50% faster in IOPS comparing to Intel X520 
 +  - Converged Network Adapter (FCoE offloading, iSCSI offload, VIRT offloading) 
 +  - Broadcom Emulex OneConnect OCe11102 ( Chipset: BladeEngine 3 be3) 
  
-Emulex OneConnect OCe11102-N 10Gb Ethernet (10GbE) 
  
 <code> <code>
Line 7: Line 15:
 04:00.1 Ethernet controller: Emulex Corporation OneConnect 10Gb NIC (be3) (rev 02) 04:00.1 Ethernet controller: Emulex Corporation OneConnect 10Gb NIC (be3) (rev 02)
 </code> </code>
 +
 +===== Issues =====
 +
 +==== AMD-Vi ====
  
 On AMD Ryzen with X570 chipset: On AMD Ryzen with X570 chipset:
Line 16: Line 28:
 </code> </code>
  
-Workaround: +**Workaround:** 
-add to kernel cmdline: ''iommu=soft''+  - add to kernel cmdline: ''iommu=soft'' 
 +  - [[https://forum.proxmox.com/threads/pci-passthrough-amd-vi-completion-wait-loop-timed-out.75832/|PCI Passthrough: AMD-Vi: Completion-Wait loop timed out]] 
 + 
 +==== bond and bridge ==== 
 + 
 +There is issue where Emulex is a bond0 slave and bond0 is part of a bridge. 
 + 
 +More here: [[vm:proxmox:issues:no_dhcp_offer]] 
 + 
 +**Solution:** upgrade to latest firmware. 
 + 
 + 
 +===== Firmware ===== 
 + 
 +==== IBM ==== 
 + 
 +Latest published (from 2011): [[https://www.ibm.com/support/pages/support-cd-release-emulex-based-10gbe-v5142-ibm-system-x| 
 +Support CD Release for Emulex-based 10GbE v5.1.42 - IBM System x 
 +]] 
 + 
 + 
 +==== Broadcom ==== 
 + 
 +Please read user guide:  
 +  - [[https://docs.broadcom.com/doc/12398083|Emulex® Drivers for Linux for OneConnect® 
 +Adapters - User Guide v 12.6]] 
 +  - https://docs.broadcom.com/doc/12378876|Emulex® Drivers for Linux for OneConnect® 
 +Adapters - User Guide v 11.2]] 
 + 
 + 
 +Download firmware upgrade from: 
 +  - [[https://www.broadcom.com/|Broadcom]] page. Look for ''OneConnect Flash ISO Image'' 
 + 
 + 
 +https://www.dell.com/support/home/pl-pl/drivers/driversdetails?driverid=1dhk6 
 + 
 +Issues 
 + 
 +[[https://community.hpe.com/t5/networking/frame-error-with-driver-be2net-amp-emulex-oneconnect-10gb/td-p/4792712|Frame error with driver be2net & Emulex OneConnect 10Gb]] 
 + 
 +=== fw versions === 
 +  - 4.1.455.10 - on old i7 machine - bond and bridge works OK 
 +  - 4.6.281.21 - on old i5 machine - bond and bridge works OK 
 +  - 4.6.281.8 - on AMD Ryzne - bond and bridge broadcast issue present 
 +  - 11.4.1186.4 - latest firmware - works OK 
 + 
 +=== fw upgrade === 
 + 
 +<code bash> 
 +ethtool -i enp37s0f0 
 + 
 +driver: be2net 
 +version: 5.15.30-2-pve 
 +firmware-version: 4.6.281.8 
 +expansion-rom-version:  
 +bus-info: 0000:25:00.0 
 +supports-statistics: yes 
 +supports-test: yes 
 +supports-eeprom-access: yes 
 +supports-register-dump: no 
 +supports-priv-flags: yes 
 + 
 +</code> 
 + 
 +Flashin firmware with ethtool. Note - process takes long time~ 
 + 
 +<code bash> 
 +cp oc11-11.4.1186.4.ufi /lib/firmware/ 
 +ethtool --flash enp37s0f0 oc11-11.4.1186.4.ufi 
 +ethtool --flash enp37s0f1 oc11-11.4.1186.4.ufi 
 +</code> 
 + 
 +<code bash> 
 +# ethtool -i enp37s0f0  
 +driver: be2net 
 +version: 5.15.30-2-pve 
 +firmware-version: 4.6.281.8 [11.4.1186.4] 
 +expansion-rom-version:  
 +bus-info: 0000:25:00.0 
 +supports-statistics: yes 
 +supports-test: yes 
 +supports-eeprom-access: yes 
 +supports-register-dump: no 
 +supports-priv-flags: yes 
 +</code>
  
-[[https://forum.proxmox.com/threads/pci-passthrough-amd-vi-completion-wait-loop-timed-out.75832/]]+Apply new firmware - reboot the system.
  
  
-Firmware upgrade on [[https://www.broadcom.com/|Broadcom]] page. Look for ''OneConnect Flash ISO Image''