You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-09 - 09:25

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot4s.osadl.org (updated Thu Jul 09, 2026 00:45:09)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
579299644,38cyclictest23242-21rm21:45:142
579299576,40cyclictest30792-21ssh22:25:142
5792995537,10cyclictest27-21ksoftirqd/220:30:142
577499474,4cyclictest131rcu_preempt22:40:130
578299462,7cyclictest131rcu_preempt19:40:121
579299453,37cyclictest18322-21ssh23:30:142
577499433,7cyclictest131rcu_preempt22:15:120
5774994226,3cyclictest12-21ksoftirqd/023:50:110
578299382,29cyclictest3873-21ssh23:40:141
580099376,23cyclictest35966irq/58-eth000:15:113
580099373,22cyclictest35966irq/58-eth022:15:113
577499371,33cyclictest28518-21ssh21:30:130
578299363,29cyclictest25521-21ssh22:40:131
580099352,16cyclictest35966irq/58-eth000:10:133
578299353,26cyclictest18112-21ssh22:00:151
577499332,2cyclictest131rcu_preempt23:20:130
579299323,23cyclictest0-21swapper/223:46:592
579299323,16cyclictest630-21Async09:36:312
5792993215,11cyclictest27-21ksoftirqd/222:50:132
577499321,19cyclictest5054-21ssh21:35:110
579299305,15cyclictest27346irq/40-dwc2_hso23:10:112
579299304,16cyclictest27346irq/40-dwc2_hso23:15:152
579299303,21cyclictest0-21swapper/223:35:132
579299302,21cyclictest0-21swapper/219:45:122
579299302,21cyclictest0-21swapper/219:30:152
579299302,21cyclictest0-21swapper/219:30:142
579299302,16cyclictest7577-21ssh00:00:142
579299302,16cyclictest7577-21ssh00:00:132
577499302,20cyclictest25471-21ssh00:10:120
580099298,16cyclictest0-21swapper/323:20:133
580099292,21cyclictest35966irq/58-eth019:40:133
580099291,19cyclictest35966irq/58-eth023:55:123
579299295,14cyclictest27346irq/40-dwc2_hso23:20:132
579299293,20cyclictest16425-21ssh22:35:122
579299292,21cyclictest28846-21ssh23:52:592
577499292,21cyclictest20637-21sed00:25:170
579299286,12cyclictest27346irq/40-dwc2_hso19:15:122
579299285,13cyclictest27346irq/40-dwc2_hso20:05:132
579299285,13cyclictest27346irq/40-dwc2_hso19:55:112
579299283,14cyclictest27346irq/40-dwc2_hso21:00:132
5782992820,4cyclictest21-21ksoftirqd/121:15:141
577499282,23cyclictest2046-21ssh00:15:100
579299276,18cyclictest27346irq/40-dwc2_hso20:55:112
579299272,20cyclictest27956-21sshd00:10:202
579299272,20cyclictest0-21swapper/219:10:142
579299272,11cyclictest24288-21diskmemload00:28:282
578299272,20cyclictest21874-21ssh23:50:131
578299272,20cyclictest13839-21ssh00:39:181
578299272,19cyclictest4833-21ssh00:34:161
578299272,19cyclictest20636-21cut00:25:181
5782992713,3cyclictest24288-21diskmemload00:10:181
580099262,18cyclictest0-21swapper/300:23:273
579299263,16cyclictest27346irq/40-dwc2_hso00:05:132
579299262,9cyclictest9382-21ssh23:41:172
5792992619,4cyclictest27-21ksoftirqd/223:25:112
579299261,20cyclictest29117-21sh23:00:132
578299262,18cyclictest32303-21ssh21:50:141
577499262,19cyclictest4236-21ssh00:33:340
580099253,15cyclictest0-21swapper/300:39:183
580099253,14cyclictest35966irq/58-eth022:40:133
5800992510,10cyclictest35966irq/58-eth021:30:153
579299255,15cyclictest27346irq/40-dwc2_hso00:20:122
579299253,14cyclictest27346irq/40-dwc2_hso20:00:142
579299253,13cyclictest27346irq/40-dwc2_hso19:25:112
579299252,9cyclictest7360-21ssh22:30:142
579299252,16cyclictest0-21swapper/220:50:052
578299252,14cyclictest20086-21ssh22:55:141
578299251,19cyclictest2091-21ssh22:45:111
578299251,19cyclictest0-21swapper/123:20:121
580099249,10cyclictest0-21swapper/322:00:123
580099243,14cyclictest0-21swapper/300:25:173
579299243,13cyclictest27346irq/40-dwc2_hso21:05:112
579299242,13cyclictest27346irq/40-dwc2_hso22:45:122
578299249,3cyclictest24288-21diskmemload21:46:381
578299249,10cyclictest21-21ksoftirqd/123:55:151
578299242,14cyclictest27278-21ssh23:35:121
5782992410,7cyclictest24288-21diskmemload23:49:311
580099233,13cyclictest0-21swapper/323:25:043
579299232,16cyclictest0-21swapper/223:57:162
579299232,16cyclictest0-21swapper/200:31:172
579299232,16cyclictest0-21swapper/200:18:172
5792992315,5cyclictest27208-21ssh22:05:142
5792992310,10cyclictest24821-21sh21:10:162
5782992312,3cyclictest24288-21diskmemload23:06:221
578299231,16cyclictest0-21swapper/121:30:151
577499232,7cyclictest12-21ksoftirqd/021:00:010
577499232,14cyclictest30782-21ssh22:25:130
5774992316,5cyclictest12-21ksoftirqd/022:00:120
577499231,19cyclictest16902-21ls20:25:220
577499231,17cyclictest11048-21ssh23:43:160
580099223,13cyclictest0-21swapper/300:34:163
579299228,5cyclictest23676-21kthreadcore22:20:192
579299225,11cyclictest27346irq/40-dwc2_hso21:50:132
579299224,12cyclictest27346irq/40-dwc2_hso21:55:122
579299222,12cyclictest0-21swapper/220:35:122
579299222,11cyclictest16340-21kthreadcore21:40:202
579299222,10cyclictest20087-21ssh22:55:142
579299221,9cyclictest7183-21latency22:10:222
579299221,6cyclictest26983-21kthreadcore22:40:202
579299221,18cyclictest8112-21kthreadcore23:05:222
579299221,13cyclictest1394-21ssh21:15:142
579299221,12cyclictest19074-21idleruntime-cro21:25:022
579299221,10cyclictest1270-21ls19:35:242
5792992210,9cyclictest0-21swapper/220:40:222
5782992212,6cyclictest0-21swapper/120:10:121
5782992212,6cyclictest0-21swapper/120:10:111
578299221,11cyclictest7088-21kthreadcore21:35:201
5774992214,5cyclictest24288-21diskmemload23:10:130
580099216,11cyclictest0-21swapper/321:15:153
5800992113,5cyclictest35966irq/58-eth023:10:133
5800992112,7cyclictest0-21swapper/322:30:173
5800992112,3cyclictest35966irq/58-eth022:50:143
5800992110,9cyclictest0-21swapper/321:53:213
579299213,12cyclictest27346irq/40-dwc2_hso21:25:122
579299212,16cyclictest13033-21idleruntime22:15:162
579299211,11cyclictest30001-21cut21:30:212
579299211,10cyclictest8480-21kthreadcore21:35:212
578299213,12cyclictest0-21swapper/119:20:121
578299211,17cyclictest13638-21ls20:55:281
578299211,16cyclictest5541-21ls00:15:191
578299211,16cyclictest16413-21apt-get22:35:111
578299211,14cyclictest3761-21ssh22:10:141
578299211,14cyclictest21771-21sh22:20:131
578299211,11cyclictest27294-21kthreadcore19:30:221
578299211,11cyclictest27294-21kthreadcore19:30:211
577499218,4cyclictest131rcu_preempt22:15:010
577499212,12cyclictest11350-1kworker/0:1H23:25:040
5774992112,7cyclictest9804-21cron20:55:020
577499211,17cyclictest23214-21acpi21:45:110
577499211,14cyclictest21770-21ssh22:20:120
577499210,16cyclictest32607-21ls20:45:020
580099209,9cyclictest0-21swapper/321:35:073
580099203,12cyclictest0-21swapper/322:20:133
579299207,3cyclictest0-21swapper/219:10:002
579299204,5cyclictest16073-21irqrtprio19:20:182
579299201,9cyclictest19806-21tr22:00:202
5792992013,3cyclictest27-21ksoftirqd/220:20:132
579299201,15cyclictest31839-21irqcore20:10:172
579299201,15cyclictest31839-21irqcore20:10:172
579299201,13cyclictest27346irq/40-dwc2_hso19:50:112
578299207,2cyclictest131rcu_preempt20:20:201
578299203,2cyclictest131rcu_preempt23:10:211
578299203,2cyclictest131rcu_preempt19:10:301
578299202,12cyclictest29116-21ssh23:00:131
578299201,5cyclictest20263-21cut23:30:201
5782992013,5cyclictest15912-21cat00:20:311
578299201,2cyclictest0-21swapper/122:17:541
578299201,10cyclictest0-21swapper/121:25:021
578299200,4cyclictest131rcu_preempt21:14:311
577499209,3cyclictest131rcu_preempt21:55:170
577499202,12cyclictest24463-21ssh00:09:150
577499201,15cyclictest12873-21apt-get23:45:120
5774992010,7cyclictest19342-21cat19:20:280
580099199,8cyclictest0-21swapper/322:29:433
580099199,6cyclictest0-21swapper/322:35:293
580099199,5cyclictest0-21swapper/321:20:133
580099193,11cyclictest0-21swapper/300:09:153
5800991913,4cyclictest0-21swapper/300:00:193
5800991913,4cyclictest0-21swapper/300:00:183
5800991910,6cyclictest35966irq/58-eth023:50:123
579299192,11cyclictest0-21swapper/220:15:132
579299191,9cyclictest365-21cpuspeed20:45:142
579299191,9cyclictest13730-21fschecks_time20:25:152
579299191,16cyclictest2582-21fschecks_count19:40:152
578299191,8cyclictest0-21swapper/122:30:531
578299191,3cyclictest0-21swapper/122:28:511
5782991912,5cyclictest16507-21ssh21:40:211
578299191,13cyclictest11099-21ssh22:50:131
578299191,10cyclictest0-21swapper/100:00:101
578299191,10cyclictest0-21swapper/100:00:091
5782991910,6cyclictest1-21systemd20:15:571
5782991910,3cyclictest24288-21diskmemload22:08:081
577499197,2cyclictest131rcu_preempt22:36:220
577499195,12cyclictest121ksoftirqd/022:30:140
577499192,14cyclictest30938-21ssh23:55:130
577499191,6cyclictest23178-21ntp_states21:05:240
5774991913,4cyclictest17910-21proc_pri21:40:260
5774991912,5cyclictest2580-21sendmail_mailqu20:10:270
5774991912,5cyclictest2580-21sendmail_mailqu20:10:260
5774991911,5cyclictest8301-21kthreadcore00:00:180
5774991911,5cyclictest8301-21kthreadcore00:00:180
577499191,14cyclictest1301-21sh21:15:150
5774991910,7cyclictest2940-21cut21:50:220
580099189,6cyclictest0-21swapper/323:36:583
580099189,6cyclictest0-21swapper/323:34:393
580099189,6cyclictest0-21swapper/323:16:193
580099189,6cyclictest0-21swapper/321:41:373
580099188,8cyclictest0-21swapper/320:40:183
580099184,11cyclictest0-21swapper/323:45:173
580099183,5cyclictest0-21swapper/323:43:163
5800991812,4cyclictest35966irq/58-eth022:10:193
580099181,14cyclictest0-21swapper/321:25:273
580099181,11cyclictest0-21swapper/323:00:293
580099181,10cyclictest0-21swapper/320:50:273
5800991810,3cyclictest0-21swapper/322:08:553
578299184,10cyclictest32450-21grep23:20:021
578299182,9cyclictest0-21swapper/100:09:151
578299182,5cyclictest21-21ksoftirqd/120:30:191
578299182,3cyclictest0-21swapper/119:15:161
578299181,9cyclictest10028-21kthreadcore21:55:171
578299181,3cyclictest0-21swapper/123:27:501
5782991811,5cyclictest17270-21sshd19:55:151
578299181,13cyclictest19448-21ssh21:25:121
577499189,6cyclictest18961-21irqrtprio23:30:180
577499188,8cyclictest12-21ksoftirqd/020:15:120
577499188,7cyclictest9955-21cat23:05:300
577499185,11cyclictest6082-21grep19:40:270
577499185,10cyclictest29863-21find19:35:020
577499185,10cyclictest29863-21find19:35:010
577499185,10cyclictest22648-21kthreadcore19:25:190
577499183,4cyclictest131rcu_preempt23:35:270
577499182,13cyclictest20948-21kthreadcore22:55:190
5774991812,4cyclictest2259-21fschecks_count22:45:140
5774991812,3cyclictest0-21swapper/020:00:140
577499181,13cyclictest9549-21ps00:35:250
577499181,12cyclictest19680-21latency_hist22:55:020
5774991810,6cyclictest23232-21grep21:25:270
5774991810,6cyclictest19494-21sshd21:04:170
5774991810,5cyclictest19759-21latency_hist00:25:020
580099179,6cyclictest0-21swapper/321:55:093
580099179,6cyclictest0-21swapper/321:45:023
580099179,5cyclictest0-21swapper/322:55:223
580099171,9cyclictest0-21swapper/319:35:013
580099171,9cyclictest0-21swapper/319:35:013
5800991712,3cyclictest33-21ksoftirqd/320:55:273
5800991712,3cyclictest33-21ksoftirqd/319:50:283
580099171,10cyclictest0-21swapper/322:45:143
578299179,5cyclictest8309-21kthreadcore20:50:211
578299171,8cyclictest0-21swapper/119:35:201
578299171,3cyclictest0-21swapper/120:45:181
578299171,13cyclictest7037-21apt-get19:45:031
578299171,13cyclictest18131-21munin-run20:30:011
578299171,12cyclictest21575-21apt-get20:00:031
577499179,5cyclictest28729-21grep21:10:280
577499173,11cyclictest6373-21sed19:10:180
577499173,11cyclictest18929-21kthreadcore19:55:190
577499173,11cyclictest10746-21ssh21:20:180
577499172,12cyclictest26740-21kthreadcore23:15:220
577499171,9cyclictest21650-21memory20:30:240
577499171,8cyclictest0-21swapper/022:07:280
577499171,14cyclictest14627-21kthreadcore19:50:200
577499171,14cyclictest13258-21munin-run20:25:020
577499171,12cyclictest10853-21diskstats19:15:150
577499171,10cyclictest11946-21latency_hist19:50:020
5774991710,5cyclictest31083-21kthreadcore23:00:220
580099168,5cyclictest35966irq/58-eth023:05:133
5800991611,3cyclictest33-21ksoftirqd/319:10:153
580099161,12cyclictest0-21swapper/319:25:273
580099161,10cyclictest0-21swapper/320:10:123
580099161,10cyclictest0-21swapper/320:10:113
5800991610,4cyclictest0-21swapper/321:10:123
5800991610,3cyclictest33-21ksoftirqd/319:20:143
578299169,5cyclictest16453-21kthreadcore21:00:191
578299168,6cyclictest23315-21cut20:35:161
578299162,11cyclictest16049-21unixbench_singl19:50:291
5782991611,3cyclictest0-21swapper/121:05:141
578299161,12cyclictest28098-21cut20:40:141
577499161,6cyclictest23332-21fschecks_time20:35:150
577499161,12cyclictest28560-21sort20:05:200
577499161,11cyclictest31102-21kthreadcore19:35:170
5774991610,4cyclictest439-21fschecks_count20:45:150
580099151,11cyclictest0-21swapper/321:05:163
578299151,11cyclictest26686-21apt-get20:05:131
578299151,10cyclictest20594-21hwlatdetect19:25:161
580099149,3cyclictest33-21ksoftirqd/320:45:273
580099149,3cyclictest33-21ksoftirqd/320:05:293
580099149,3cyclictest33-21ksoftirqd/319:15:133
580099149,3cyclictest0-21swapper/321:00:163
580099148,4cyclictest0-21swapper/320:25:593
580099142,6cyclictest0-21swapper/320:20:123
580099141,6cyclictest0-21swapper/319:35:153
578299141,10cyclictest5845-21cat19:10:021
577499141,10cyclictest5846-21latency_hist19:10:010
580099138,3cyclictest33-21ksoftirqd/320:30:153
580099138,3cyclictest33-21ksoftirqd/319:45:103
580099137,4cyclictest33-21ksoftirqd/319:55:253
580099133,7cyclictest35966irq/58-eth020:15:113
580099122,8cyclictest0-21swapper/320:35:173
580099112,3cyclictest0-21swapper/320:00:153
580099101,6cyclictest0-21swapper/319:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional