You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-04 - 21:05

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot4s.osadl.org (updated Mon May 04, 2026 12:45:10)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1438999638,7cyclictest131rcu_preempt11:00:130
1438999627,11cyclictest131rcu_preempt08:45:140
1440499601,44cyclictest25490-21ssh10:00:131
14418995617,8cyclictest131rcu_preempt07:50:122
1438999563,40cyclictest18366-21ssh09:20:120
1441899539,9cyclictest131rcu_preempt11:15:132
1438999534,34cyclictest9395-21ssh09:15:140
1441899522,41cyclictest19343-21ssh11:45:132
14431995034,10cyclictest35966irq/58-eth012:15:143
1440499502,15cyclictest11613-21ssh12:35:121
1440499484,6cyclictest131rcu_preempt11:15:131
14404994817,22cyclictest21-21ksoftirqd/109:45:131
1441899472,5cyclictest131rcu_preempt09:25:132
1443199461,28cyclictest35966irq/58-eth009:35:133
14431994435,6cyclictest35966irq/58-eth009:50:153
1440499441,34cyclictest21456-21ssh11:10:131
1441899436,24cyclictest27-21ksoftirqd/211:05:142
14404994118,12cyclictest630-21Async09:36:311
1440499394,18cyclictest29129-21ssh10:20:141
1440499382,22cyclictest21-21ksoftirqd/107:25:121
1438999381,29cyclictest3963-21rm09:30:130
14404993722,8cyclictest536-21usb-storage12:00:111
1440499371,18cyclictest18057-21ssh10:50:131
14404993616,14cyclictest21-21ksoftirqd/107:50:131
1438999362,27cyclictest22051-21ssh09:40:130
1443199352,21cyclictest35966irq/58-eth011:05:153
1440499354,13cyclictest9060-21find10:45:131
1440499352,29cyclictest16582-21ssh09:55:131
14389993523,9cyclictest0-21swapper/008:05:120
1438999352,22cyclictest12-21ksoftirqd/009:35:130
1443199349,15cyclictest321rcuc/308:45:143
14431993423,7cyclictest0-21swapper/312:05:123
1441899345,20cyclictest261rcuc/210:15:132
1438999341,20cyclictest27364-21ssh09:25:120
14431993317,10cyclictest35966irq/58-eth011:50:153
1440499331,27cyclictest10371-21sh11:40:101
1438999331,23cyclictest32406-21diskmemload12:20:120
1443199327,20cyclictest35966irq/58-eth012:35:123
1441899323,18cyclictest27346irq/40-dwc2_hso08:05:132
14404993217,9cyclictest14380-21cyclictest08:00:121
1443199313,17cyclictest35966irq/58-eth010:00:143
1443199312,18cyclictest35966irq/58-eth009:25:123
1443199311,17cyclictest35966irq/58-eth007:10:123
1440499313,20cyclictest14559-21ssh10:30:131
1440499313,20cyclictest14559-21ssh10:30:131
14389993113,14cyclictest12-21ksoftirqd/010:25:140
1443199302,13cyclictest35966irq/58-eth012:10:123
1440499302,20cyclictest22783-21ssh12:05:141
14404993011,9cyclictest14380-21cyclictest07:20:141
1438999302,22cyclictest8207-21ssh12:31:230
1441899293,8cyclictest0-21swapper/212:31:232
14418992915,6cyclictest31612-21ssh12:10:122
14418992910,9cyclictest27-21ksoftirqd/208:45:142
1438999291,24cyclictest12-21ksoftirqd/010:50:130
1438999291,20cyclictest12-21ksoftirqd/010:15:130
1443199283,18cyclictest0-21swapper/312:30:593
14431992814,8cyclictest35966irq/58-eth011:25:113
1441899283,16cyclictest27346irq/40-dwc2_hso08:25:122
1441899282,15cyclictest27346irq/40-dwc2_hso08:30:122
14418992818,7cyclictest27-21ksoftirqd/211:35:132
14418992818,7cyclictest27-21ksoftirqd/211:35:132
1440499282,20cyclictest31634-21ssh12:10:141
1441899273,17cyclictest27346irq/40-dwc2_hso07:35:142
1440499272,19cyclictest6141-21sed12:30:241
1441899262,20cyclictest28238-21sh11:50:152
1441899262,11cyclictest13013-21ssh09:35:142
1441899262,10cyclictest131rcu_preempt07:15:142
1441899261,18cyclictest27346irq/40-dwc2_hso10:20:132
1440499262,19cyclictest3613-21diskmemload11:00:141
14404992619,3cyclictest0-21swapper/107:40:131
1438999267,12cyclictest29113-21ssh10:20:130
1438999263,15cyclictest12-21ksoftirqd/011:20:120
1443199252,18cyclictest35966irq/58-eth010:35:133
1443199252,18cyclictest35966irq/58-eth010:35:133
1441899259,7cyclictest42-21kauditd08:55:012
1441899253,13cyclictest2124-21sh10:05:132
14418992511,11cyclictest14665-21memory10:10:242
14404992514,7cyclictest7205-21cron08:10:011
1438999252,18cyclictest819-21ssh12:28:230
1443199246,16cyclictest33-21ksoftirqd/310:15:033
1443199244,15cyclictest0-21swapper/310:05:133
1443199243,14cyclictest0-21swapper/309:30:133
1441899241,8cyclictest18370-21sh09:20:132
1441899241,19cyclictest12346-21cut08:15:162
14418992411,6cyclictest21841-21ssh11:10:192
1440499242,15cyclictest1983-21ssh10:05:041
1438999241,17cyclictest27075-21ssh10:55:140
1443199231,15cyclictest0-21swapper/310:55:133
1441899236,9cyclictest11312-21unixbench_singl08:10:292
1441899232,16cyclictest10368-21apt-get11:40:112
14418992311,5cyclictest23201-21ssh11:28:292
14418992311,5cyclictest14817-21ssh12:17:462
14418992311,10cyclictest17692-21ssh09:19:432
14418992310,10cyclictest1-21systemd12:36:002
1440499231,15cyclictest1407-21ssh11:35:141
1440499231,15cyclictest1407-21ssh11:35:141
1443199223,11cyclictest0-21swapper/310:30:123
1443199223,11cyclictest0-21swapper/310:30:123
1441899228,11cyclictest14698-21fschecks_count10:30:152
1441899228,11cyclictest14698-21fschecks_count10:30:152
1441899227,6cyclictest2563-21ssh09:10:222
1441899227,6cyclictest2563-21ssh09:10:222
1441899226,13cyclictest13691-21ssh10:45:322
1441899223,13cyclictest27346irq/40-dwc2_hso11:20:122
1441899222,9cyclictest131rcu_preempt09:55:142
1441899222,16cyclictest25489-21ssh10:00:132
1441899221,7cyclictest25174-21if_eth011:30:172
14418992211,9cyclictest29658-21ssh12:08:052
1441899221,11cyclictest13272-21idleruntime-cro12:00:012
14418992210,9cyclictest21910-21ssh09:40:052
1440499221,18cyclictest21829-21find08:25:111
1438999221,18cyclictest25438-21find07:55:110
1438999221,17cyclictest28573-21ssh11:30:270
14389992211,7cyclictest12-21ksoftirqd/008:25:130
1438999221,15cyclictest28240-21ssh11:50:150
1438999221,15cyclictest19349-21sh11:45:140
1443199214,9cyclictest35966irq/58-eth009:20:143
1443199212,11cyclictest35966irq/58-eth012:20:133
14431992110,8cyclictest0-21swapper/311:40:193
1441899219,9cyclictest0-21swapper/208:35:242
1441899217,5cyclictest28632-21kthreadcore12:25:212
1441899217,11cyclictest22812-21sh09:00:012
1441899213,8cyclictest31322-21tune2fs09:45:172
1441899213,12cyclictest27-21ksoftirqd/210:35:122
1441899213,12cyclictest27-21ksoftirqd/210:35:122
1441899212,11cyclictest14352-21latency07:40:222
1441899212,10cyclictest26877-21ssh10:55:022
1441899211,8cyclictest566366ssh11:00:222
14418992116,3cyclictest28271-21ssh10:55:202
14418992112,6cyclictest32465-21apt-get10:40:132
1441899211,12cyclictest19421-21kthreadcore12:20:202
1440499218,9cyclictest15831-21diskmemload11:25:121
1440499211,15cyclictest17310-21ls08:20:171
1440499211,13cyclictest27840-21find09:05:131
1440499211,13cyclictest23285-21ls09:00:161
1438999218,9cyclictest19901-21kthreadcore09:55:220
1438999215,3cyclictest131rcu_preempt12:06:450
1438999213,11cyclictest12-21ksoftirqd/012:10:120
1438999213,11cyclictest0-21swapper/007:30:050
1438999211,5cyclictest0-21swapper/012:35:180
14389992112,6cyclictest0-21swapper/011:40:160
1438999211,17cyclictest21749-21ls08:25:010
1438999211,15cyclictest361-21ssh09:10:140
1438999211,15cyclictest361-21ssh09:10:140
1438999211,10cyclictest14778-21fw_packets07:10:160
1443199205,11cyclictest0-21swapper/312:25:283
1443199202,12cyclictest0-21swapper/310:20:153
1443199201,15cyclictest35966irq/58-eth010:45:143
14431992010,8cyclictest0-21swapper/310:26:583
14431992010,8cyclictest0-21swapper/309:15:353
1441899208,9cyclictest23516-21irqcore09:00:172
1441899204,9cyclictest25510-21kthreadcore07:20:192
1441899202,15cyclictest31236-21kthreadcore07:25:212
1441899202,15cyclictest25869-21irqcore07:55:172
1441899201,9cyclictest19112-21cron07:15:002
1441899201,9cyclictest1593-21fschecks_time07:30:162
1441899201,16cyclictest17204-21sort08:20:172
1441899201,15cyclictest7062-21ssh10:25:202
1441899201,14cyclictest930-21snmpd07:45:152
1441899201,10cyclictest7975-21ps09:50:192
1441899201,10cyclictest5023-21grep09:30:192
1441899200,10cyclictest32659-21kthreadcore08:00:212
1441899200,10cyclictest32563-21latency_hist09:10:012
1440499204,10cyclictest0-21swapper/107:10:121
1440499202,13cyclictest28350-21diskstats11:50:161
1440499202,13cyclictest23532-21ssh10:35:151
1440499202,13cyclictest23532-21ssh10:35:141
1440499201,7cyclictest28238-21kthreadcore10:55:201
1440499201,15cyclictest32495-21rm10:40:141
1440499201,15cyclictest20751-21ssh12:20:251
1440499201,14cyclictest8088-21ssh12:15:131
1440499201,11cyclictest6822-21kthreadcore09:30:211
14404992010,7cyclictest9536-21memory07:35:221
1438999202,13cyclictest844-21munin-node11:15:150
14389992013,4cyclictest27413-21timerandwakeup10:35:280
14389992013,4cyclictest27413-21timerandwakeup10:35:280
1438999201,16cyclictest9101-21ssh10:45:150
1438999201,14cyclictest844-21munin-node07:45:140
1438999201,12cyclictest14540-21sh10:30:130
1438999201,12cyclictest14540-21sh10:30:120
1438999200,4cyclictest131rcu_preempt08:50:110
1443199198,6cyclictest0-21swapper/311:45:143
1443199194,6cyclictest0-21swapper/311:20:123
1443199191,13cyclictest0-21swapper/310:10:313
14431991910,6cyclictest0-21swapper/309:47:163
1441899198,8cyclictest4360-21irqrtprio08:40:192
1441899191,10cyclictest14477-21latency_hist07:10:012
1441899191,10cyclictest13844-21df12:00:132
1440499199,7cyclictest30675-21cron09:45:011
1440499193,13cyclictest18281-21sort08:55:131
1440499192,15cyclictest28156-21kthreadcore11:30:231
1440499192,13cyclictest12538-21apt-get09:35:031
1440499191,9cyclictest8640-21kthreadcore08:10:201
14404991912,5cyclictest0-21swapper/107:15:171
14404991911,5cyclictest11492-21kthreadcore09:15:221
1440499190,13cyclictest591-21cut09:10:181
1440499190,13cyclictest591-21cut09:10:171
1438999199,7cyclictest8071-21apt-get12:15:130
1438999197,9cyclictest21875-21cut11:10:180
1438999197,8cyclictest151rcuc/011:05:140
1438999191,15cyclictest3853-21apt-get08:40:130
14389991910,7cyclictest14732-21cat10:10:250
1443199189,6cyclictest35966irq/58-eth011:10:123
1443199184,9cyclictest0-21swapper/310:50:123
1443199182,7cyclictest0-21swapper/309:00:113
1443199182,6cyclictest0-21swapper/312:00:133
1443199182,10cyclictest0-21swapper/308:35:133
1443199182,10cyclictest0-21swapper/308:30:133
1443199182,10cyclictest0-21swapper/307:40:133
1443199181,11cyclictest0-21swapper/309:55:173
1443199181,11cyclictest0-21swapper/307:55:273
14431991810,6cyclictest0-21swapper/310:40:213
1440499188,8cyclictest5578-21apt-get10:25:121
1440499187,5cyclictest0-21swapper/111:05:151
1440499184,10cyclictest18748-21kthreadcore07:45:211
1440499181,7cyclictest8931-21kthreadcore11:20:201
14404991812,4cyclictest0-21swapper/110:13:411
1440499181,13cyclictest5456-21irqstats11:55:191
14404991810,6cyclictest3595-21sshd08:39:231
14404991810,5cyclictest22330-21cat09:20:281
1438999186,3cyclictest131rcu_preempt09:45:080
1438999184,8cyclictest12-21ksoftirqd/009:00:120
1438999182,12cyclictest1513-21df_inode11:35:170
1438999182,12cyclictest1513-21df_inode11:35:160
1438999181,5cyclictest22856-21cut09:00:010
1438999181,15cyclictest2441-21sshd08:35:230
14389991811,5cyclictest0-21swapper/010:05:030
1438999181,13cyclictest7701-21df_inode09:50:160
1443199179,6cyclictest0-21swapper/311:55:443
1443199179,6cyclictest0-21swapper/311:30:113
1443199179,6cyclictest0-21swapper/311:00:113
1443199179,6cyclictest0-21swapper/309:41:223
1443199178,6cyclictest35966irq/58-eth011:35:153
1443199178,6cyclictest35966irq/58-eth011:35:143
1443199173,6cyclictest0-21swapper/307:50:123
1443199172,11cyclictest0-21swapper/308:50:113
1443199172,11cyclictest0-21swapper/308:40:163
1443199172,10cyclictest0-21swapper/308:10:153
1443199171,13cyclictest35966irq/58-eth011:15:153
1440499179,6cyclictest17985-21sshd08:52:231
1440499178,7cyclictest32079-21cat09:25:311
1440499171,12cyclictest25802-21irqcore07:55:161
1440499171,12cyclictest21843-21kthreadcore11:45:221
1440499171,11cyclictest29733-21ssh12:25:261
14404991710,4cyclictest20507-21interrupts10:15:171
1438999179,5cyclictest9529-21kthreadcore08:10:220
1438999178,3cyclictest12-21ksoftirqd/011:55:030
1438999172,6cyclictest2363-21perl10:05:160
1438999171,7cyclictest20672-21apt-get07:50:110
1438999171,5cyclictest0-21swapper/011:25:480
1438999171,3cyclictest0-21swapper/007:35:120
1438999171,2cyclictest0-21swapper/012:00:270
14389991710,5cyclictest15624-21sshd07:44:220
14389991710,4cyclictest2319-21cut10:40:220
1438999170,14cyclictest29931-21kthreadcore09:05:210
1443199163,9cyclictest0-21swapper/309:10:153
1443199163,9cyclictest0-21swapper/309:10:153
1443199162,9cyclictest0-21swapper/307:45:143
1443199162,9cyclictest0-21swapper/307:35:133
1443199162,9cyclictest0-21swapper/307:25:133
1440499164,9cyclictest27063-21irqcore08:30:171
1440499163,10cyclictest6379-21kthreadcore08:40:221
1440499162,11cyclictest2031-21irqrtprio07:30:191
1440499161,12cyclictest8744-21grep08:45:171
14404991610,4cyclictest15797-21sshd08:15:271
1438999169,4cyclictest19464-21cut07:15:170
1438999168,6cyclictest30477-21kthreadcore07:25:190
1438999161,12cyclictest30387-21fschecks_count08:00:160
14389991610,3cyclictest12-21ksoftirqd/008:30:110
1443199152,10cyclictest0-21swapper/309:05:143
1443199151,8cyclictest0-21swapper/308:25:013
14431991510,3cyclictest0-21swapper/308:25:243
14431991510,3cyclictest0-21swapper/308:17:233
1438999152,11cyclictest14380-21cyclictest07:10:010
1438999152,10cyclictest12237-21apt-get08:15:120
1438999151,11cyclictest25977-21tr07:20:200
1443199149,3cyclictest0-21swapper/308:08:233
1443199141,8cyclictest0-21swapper/307:20:143
1443199141,6cyclictest0-21swapper/307:20:023
1443199141,11cyclictest0-21swapper/307:35:013
1443199141,10cyclictest0-21swapper/307:10:023
1440499141,10cyclictest1-21systemd07:10:011
1443199132,5cyclictest0-21swapper/308:00:163
1443199131,7cyclictest0-21swapper/308:55:153
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional