You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-27 - 05:58

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot3s.osadl.org (updated Wed May 27, 2026 00:44:30)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
86022240164,24sleep20-21swapper/219:08:282
84612236163,24sleep10-21swapper/119:06:381
86532233164,57sleep30-21swapper/319:09:103
86152233161,59sleep00-21swapper/019:08:380
2856321450,4sleep1880999cyclictest22:20:131
8814998481,2cyclictest0-21swapper/220:03:352
231542780,4sleep323158-21gltestperf23:20:153
231542780,4sleep323158-21gltestperf23:20:153
94342710,5sleep39433-21gltestperf21:35:163
127492600,3sleep121-21rcuc/120:30:131
180312570,1sleep10-21swapper/100:20:251
8814995547,7cyclictest0-21swapper/221:17:312
157512530,10sleep10-21swapper/121:50:151
8814995249,2cyclictest0-21swapper/219:49:172
50582510,4sleep230-21rcuc/221:25:172
108432480,1sleep00-21swapper/022:50:270
8814994744,2cyclictest0-21swapper/220:09:002
115962450,1sleep00-21swapper/022:55:120
9662430,2sleep10-21swapper/121:15:211
310912360,1sleep20-21swapper/221:10:192
41132350,1sleep30-21swapper/323:50:173
880299344,14cyclictest6954-21munin-run22:45:020
881499321,15cyclictest0-21swapper/200:10:182
283822320,3sleep1880999cyclictest22:20:011
8814993128,2cyclictest0-21swapper/221:23:582
881499310,24cyclictest0-21swapper/222:18:192
880999311,13cyclictest0-21swapper/120:39:211
881499301,14cyclictest0-21swapper/221:34:142
880299305,12cyclictest14201-21awk00:15:000
8814992917,2cyclictest0-21swapper/223:05:252
8814992916,3cyclictest0-21swapper/222:43:332
881499291,14cyclictest0-21swapper/222:10:362
881499291,14cyclictest0-21swapper/222:10:352
881499291,13cyclictest0-21swapper/219:26:402
881499291,13cyclictest0-21swapper/219:12:072
881499291,12cyclictest0-21swapper/222:46:312
881499291,12cyclictest0-21swapper/219:51:112
881499291,12cyclictest0-21swapper/219:51:112
881499290,13cyclictest0-21swapper/222:55:192
881499281,13cyclictest0-21swapper/200:40:002
881499281,13cyclictest0-21swapper/200:16:362
881499281,12cyclictest9720-40dnf00:04:242
881499281,12cyclictest0-21swapper/223:15:092
881499281,12cyclictest0-21swapper/220:56:472
880999280,14cyclictest0-21swapper/123:55:171
881499271,14cyclictest0-21swapper/222:03:202
881499271,13cyclictest0-21swapper/221:58:482
881499271,12cyclictest0-21swapper/223:32:032
881499271,12cyclictest0-21swapper/221:36:482
881499271,12cyclictest0-21swapper/220:54:432
881499271,12cyclictest0-21swapper/219:57:192
881499271,11cyclictest0-21swapper/220:15:592
881499271,11cyclictest0-21swapper/219:16:032
881499271,10cyclictest0-21swapper/200:30:142
8814992710,4cyclictest17276-21df20:40:132
881499270,13cyclictest0-21swapper/221:48:082
880999271,11cyclictest0-21swapper/123:50:421
880999270,12cyclictest0-21swapper/122:53:531
882099267,12cyclictest0-21swapper/300:38:593
881499269,3cyclictest0-21swapper/220:29:152
881499261,12cyclictest22224-21expr22:05:122
881499261,12cyclictest0-21swapper/223:51:322
881499260,13cyclictest0-21swapper/222:38:152
880999261,11cyclictest0-21swapper/119:48:451
880999260,12cyclictest0-21swapper/123:18:221
880299268,15cyclictest10993-21idleruntime-cro19:15:010
880299263,11cyclictest16726-21idleruntime-cro20:40:010
129822260,4sleep0880299cyclictest21:45:000
882099251,19cyclictest0-21swapper/322:26:263
881499259,3cyclictest0-21swapper/223:28:512
881499251,20cyclictest2186-21vmstat22:30:252
881499251,10cyclictest0-21swapper/222:28:152
881499251,10cyclictest0-21swapper/219:35:322
8814992510,3cyclictest0-21swapper/223:56:042
8814992510,3cyclictest0-21swapper/223:48:362
8814992510,3cyclictest0-21swapper/223:37:592
8814992510,3cyclictest0-21swapper/223:22:152
8814992510,3cyclictest0-21swapper/223:22:152
8814992510,3cyclictest0-21swapper/223:11:272
8814992510,3cyclictest0-21swapper/200:28:322
8809992512,11cyclictest6026-21kworker/u8:120:58:241
880999250,11cyclictest0-21swapper/119:14:411
882099241,10cyclictest9444-21cpuspeed_turbos22:50:133
881499241,13cyclictest0-21swapper/221:44:562
881499241,11cyclictest14431-21df19:20:142
881499241,10cyclictest0-21swapper/220:10:272
881499241,10cyclictest0-21swapper/200:06:322
8814992410,3cyclictest0-21swapper/222:24:032
880999246,8cyclictest0-21swapper/121:41:221
882099237,8cyclictest0-21swapper/323:05:263
882099231,6cyclictest0-21swapper/322:39:503
882099231,11cyclictest5040-21cpuspeed_turbos22:40:113
881499239,3cyclictest0-21swapper/220:33:272
881499238,3cyclictest0-21swapper/223:01:112
8814992315,3cyclictest0-21swapper/221:02:132
8814992315,3cyclictest0-21swapper/221:02:132
8814992314,3cyclictest0-21swapper/220:22:532
881499231,14cyclictest29944-21latency_hist21:10:012
881499231,13cyclictest0-21swapper/223:40:262
881499230,15cyclictest0-21swapper/220:45:212
880999239,3cyclictest30185-21sh19:55:141
8809992310,8cyclictest0-21swapper/121:55:221
882099221,6cyclictest0-21swapper/321:25:433
881499229,3cyclictest0-21swapper/222:51:392
881499221,11cyclictest0-21swapper/221:52:562
881499220,12cyclictest0-21swapper/200:20:082
8809992217,3cyclictest19482-21df_inode20:45:141
880999221,15cyclictest0-21swapper/123:05:351
8809992211,3cyclictest7374-21expr22:45:121
880999221,10cyclictest2782-21/usr/sbin/munin22:35:231
880999220,16cyclictest0-21swapper/123:45:151
880299227,5cyclictest0-21swapper/021:22:580
880299227,13cyclictest28341-21awk22:20:010
880299225,9cyclictest7-21ksoftirqd/019:30:110
880299225,15cyclictest28930-21idleruntime-cro23:35:010
880299225,14cyclictest10141-21basename20:25:020
882099211,11cyclictest15356-21/usr/sbin/munin21:50:153
8809992114,2cyclictest0-21swapper/123:20:351
8809992114,2cyclictest0-21swapper/123:20:351
8809992113,3cyclictest0-21swapper/122:57:101
8809992113,2cyclictest0-21swapper/122:04:001
880999211,18cyclictest0-21swapper/119:35:141
880999211,18cyclictest0-21swapper/119:15:161
880999211,15cyclictest0-21swapper/123:28:201
880999211,15cyclictest0-21swapper/120:13:011
880999211,14cyclictest0-21swapper/123:33:461
880999211,14cyclictest0-21swapper/120:15:111
880999211,14cyclictest0-21swapper/120:06:381
880999210,16cyclictest0-21swapper/120:04:401
880999210,15cyclictest0-21swapper/100:11:321
880299217,5cyclictest0-21swapper/023:49:180
880299217,5cyclictest0-21swapper/000:33:380
880299216,2cyclictest11-21rcuc/000:15:200
880299215,9cyclictest0-21swapper/022:07:590
880299214,3cyclictest28933-21head21:05:200
880299212,10cyclictest13157-21/usr/sbin/munin21:45:140
880299211,6cyclictest0-21swapper/023:19:520
880299211,11cyclictest31710-21/usr/sbin/munin20:00:270
882099208,3cyclictest6556-21head20:15:153
882099204,9cyclictest39-21rcuc/319:40:153
882099202,10cyclictest10014-21/usr/sbin/munin00:05:293
882099201,7cyclictest0-21swapper/322:20:173
882099201,6cyclictest0-21swapper/300:32:483
882099201,5cyclictest39-21rcuc/323:10:173
8820992013,4cyclictest24903-21tr23:25:123
882099201,11cyclictest839-21lldpd20:27:503
882099201,10cyclictest12475-21sed21:40:253
882099201,10cyclictest0-21swapper/320:55:153
8814992012,3cyclictest0-21swapper/220:35:492
881499201,13cyclictest0-21swapper/219:31:242
880999201,3cyclictest10707-21hwlatdetect00:05:191
8809992013,2cyclictest0-21swapper/120:43:431
8809992012,3cyclictest0-21swapper/122:26:271
880999201,14cyclictest0-21swapper/122:41:161
880999201,14cyclictest0-21swapper/121:49:151
880999201,14cyclictest0-21swapper/121:30:561
880999201,14cyclictest0-21swapper/121:23:031
880999201,14cyclictest0-21swapper/121:08:221
880999201,14cyclictest0-21swapper/119:52:041
880999201,14cyclictest0-21swapper/119:52:041
880999201,14cyclictest0-21swapper/100:25:041
880999201,14cyclictest0-21swapper/100:04:441
880999201,11cyclictest32110-21/usr/sbin/munin21:15:021
880999201,11cyclictest27181-21timerwakeupswit21:00:261
880999201,11cyclictest27181-21timerwakeupswit21:00:261
880999201,11cyclictest22390-21ntpq20:50:221
880999200,15cyclictest0-21swapper/122:11:311
880999200,15cyclictest0-21swapper/122:11:311
880999200,14cyclictest87650irq/129-enp1s0-23:41:431
880999200,14cyclictest0-21swapper/120:22:281
880299206,6cyclictest0-21swapper/022:14:160
880299206,6cyclictest0-21swapper/022:14:160
880299206,6cyclictest0-21swapper/000:00:460
880299206,11cyclictest11-21rcuc/023:55:110
880299205,6cyclictest0-21swapper/023:05:310
880299204,3cyclictest30740-21head21:10:160
880299201,9cyclictest0-21swapper/000:26:310
8802992012,3cyclictest0-21swapper/023:50:560
880299201,11cyclictest485-21/usr/sbin/munin22:30:170
880299201,11cyclictest4365-21/usr/sbin/munin21:25:220
880299201,11cyclictest18180-21/usr/sbin/munin23:10:220
880299201,10cyclictest0-21swapper/021:30:020
880299200,9cyclictest0-21swapper/023:35:550
238122201,2sleep10-21swapper/122:08:381
882099199,3cyclictest16361-21kworker/3:223:45:253
882099192,14cyclictest24797-21cut22:10:163
882099192,14cyclictest24797-21cut22:10:163
882099191,7cyclictest28883-21unixbench_singl19:50:273
882099191,7cyclictest28883-21unixbench_singl19:50:273
882099191,5cyclictest21301-21grep22:00:293
882099191,5cyclictest0-21swapper/300:15:483
882099191,11cyclictest0-21swapper/322:05:273
882099191,10cyclictest31357-21/usr/sbin/munin23:40:183
882099191,10cyclictest18805-21/usr/sbin/munin00:25:183
882099191,10cyclictest14095-21/usr/sbin/munin19:20:183
881499191,14cyclictest0-21swapper/219:40:322
8809991915,3cyclictest0-21swapper/119:31:331
880999191,3cyclictest87650irq/129-enp1s0-21:35:201
8809991911,3cyclictest0-21swapper/121:28:421
8809991911,3cyclictest0-21swapper/100:35:131
8809991911,3cyclictest0-21swapper/100:17:101
880999191,12cyclictest0-21swapper/100:30:141
880999191,10cyclictest18387-21cstates23:10:131
880999190,14cyclictest0-21swapper/120:27:271
880299196,3cyclictest790-21in:imjournal21:39:220
880299195,5cyclictest0-21swapper/021:53:070
880299191,6cyclictest0-21swapper/022:49:070
880299191,15cyclictest22476-21seq20:50:230
880299191,11cyclictest0-21swapper/021:55:210
880299191,10cyclictest3753-21/usr/sbin/munin20:10:150
880299191,10cyclictest28554-21/usr/sbin/munin22:20:150
880299190,16cyclictest4763-21awk22:40:020
273882191,3sleep00-21swapper/021:01:040
273882191,3sleep00-21swapper/021:01:030
179232191,3sleep10-21swapper/119:25:331
882099188,7cyclictest27092-21ls19:50:013
882099188,2cyclictest12241-21cpu00:10:123
882099187,2cyclictest0-21swapper/323:03:193
882099186,4cyclictest17329-21basename00:20:163
882099182,15cyclictest0-21swapper/323:39:463
882099182,13cyclictest622-21tr22:30:123
882099181,9cyclictest27525-21ps22:15:223
882099181,8cyclictest7817-21cpu00:00:123
882099181,7cyclictest0-21swapper/321:19:493
882099181,6cyclictest22696-21munin-run19:40:003
882099181,5cyclictest9915-21seq19:10:183
882099181,5cyclictest17895-21/usr/sbin/munin21:55:133
8820991814,3cyclictest0-21swapper/321:11:263
8820991812,3cyclictest21431-21grep20:50:133
8820991811,4cyclictest26258-21grep21:00:163
8820991811,4cyclictest26258-21grep21:00:163
8820991811,4cyclictest15573-21kernelversion20:35:193
882099181,11cyclictest0-21swapper/321:49:253
882099181,10cyclictest20759-21fschecks_count23:15:143
880999181,6cyclictest910-21systemd-logind19:40:181
8809991815,2cyclictest0-21swapper/119:22:511
880999181,4cyclictest13861-21ls23:00:111
8809991810,2cyclictest87650irq/129-enp1s0-22:30:161
880999180,17cyclictest0-21swapper/123:35:531
880299187,5cyclictest0-21swapper/023:29:240
880299185,5cyclictest0-21swapper/023:00:480
880299184,3cyclictest87550irq/128-enp1s0-19:40:180
880299183,5cyclictest0-21swapper/022:28:170
880299181,9cyclictest808-21snmpd19:19:250
880299181,9cyclictest773-21NetworkManager19:47:160
880299181,9cyclictest31183-21sshd19:56:190
880299181,3cyclictest3068-21grep20:05:260
8802991812,3cyclictest18307-21munin-run19:30:000
880299181,11cyclictest0-21swapper/021:18:130
880299181,10cyclictest0-21swapper/000:35:170
8802991810,4cyclictest827-21runrttasks20:15:140
880299180,8cyclictest0-21swapper/023:20:090
880299180,8cyclictest0-21swapper/023:20:090
880299180,5cyclictest0-21swapper/023:41:500
201722181,2sleep30-21swapper/319:30:563
882099173,11cyclictest39-21rcuc/321:30:173
882099171,6cyclictest30492-21sed19:55:193
882099171,6cyclictest0-21swapper/320:45:143
882099171,3cyclictest18175-21runrttasks19:28:303
8820991713,3cyclictest0-21swapper/323:30:533
8820991712,3cyclictest6293-21idleruntime23:55:163
882099171,10cyclictest0-21swapper/321:05:143
882099171,10cyclictest0-21swapper/320:10:223
8820991710,4cyclictest909-21sssd_nss21:20:143
8820991710,4cyclictest12744-21/usr/sbin/munin19:15:173
880299175,3cyclictest0-21swapper/022:02:570
880299171,6cyclictest0-21swapper/000:06:550
8802991711,3cyclictest1537-21crond19:25:010
880299171,10cyclictest0-21swapper/000:20:030
882099163,9cyclictest1-21systemd20:02:233
882099161,9cyclictest0-21swapper/322:45:153
882099161,4cyclictest0-21swapper/322:58:023
882099161,14cyclictest0-21swapper/320:42:443
8820991611,3cyclictest13496-21missed_timers20:30:193
8802991611,3cyclictest27551-21cat19:50:130
8802991611,3cyclictest27551-21cat19:50:130
880299161,12cyclictest10352-21/usr/sbin/munin20:25:130
8802991610,4cyclictest20727-21/usr/sbin/munin19:35:140
100632160,2sleep30-21swapper/320:24:223
882099151,11cyclictest2419-21sshd20:05:193
880299152,11cyclictest16940-21/usr/sbin/munin20:40:230
880299151,5cyclictest23878-21diskstats20:55:150
880299151,2cyclictest0-21swapper/020:30:170
8802991510,3cyclictest19325-21expr20:45:130
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional