You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-02 - 10:54

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot3s.osadl.org (updated Thu Jul 02, 2026 00:44:29)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
113612233161,59sleep00-21swapper/019:05:080
132032230162,55sleep20-21swapper/219:08:272
130642204166,25sleep30-21swapper/319:06:383
131872202166,23sleep10-21swapper/119:08:131
1334121840,3sleep01339799cyclictest22:45:270
1334121840,3sleep01339799cyclictest22:45:270
2681821780,3sleep11340599cyclictest20:50:191
1934321630,4sleep01339799cyclictest00:15:130
2043421470,4sleep31341799cyclictest00:15:253
2465621340,5sleep31341799cyclictest23:14:203
13413991080,106cyclictest0-21swapper/200:32:502
13413998582,2cyclictest0-21swapper/219:53:082
277942590,2sleep230-21rcuc/219:40:122
108232590,4sleep210829-21gltestperf23:55:162
108232590,4sleep210829-21gltestperf23:55:162
202742520,4sleep230-21rcuc/200:15:232
257642490,1sleep09-21rcu_preempt20:50:010
1339799399,14cyclictest15927-21munin-run22:55:010
1339799358,12cyclictest8719-21perf21:25:000
1339799346,13cyclictest27874-21munin-run20:55:010
1339799329,11cyclictest3705-21perf20:00:010
13413993128,2cyclictest0-21swapper/222:09:042
1341799291,13cyclictest6230-21/usr/sbin/munin22:30:213
1341799291,10cyclictest0-21swapper/300:06:063
1340599295,6cyclictest17286-40dnf21:42:261
1339799296,5cyclictest0-21swapper/023:21:030
1339799292,8cyclictest12241-21kworker/0:020:18:400
1341799281,16cyclictest0-21swapper/320:31:273
1341799281,13cyclictest0-21swapper/321:06:393
1341799280,12cyclictest0-21swapper/323:41:373
1341399281,7cyclictest3709-21cpuspeed_turbos23:40:112
1340599281,16cyclictest13699-21/usr/sbin/munin21:35:151
1340599281,16cyclictest13699-21/usr/sbin/munin21:35:151
13405992810,4cyclictest4002awk21:05:131
1339799281,20cyclictest0-21swapper/022:07:530
1339799281,19cyclictest0-21swapper/021:26:100
1339799280,13cyclictest9336-21idleruntime-cro22:40:010
213002270,5sleep321304-21latency_hist20:40:013
13417992711,3cyclictest0-21swapper/320:18:433
1341799271,13cyclictest0-21swapper/319:29:593
1341799271,13cyclictest0-21swapper/300:26:193
1341799271,12cyclictest0-21swapper/323:32:473
1341799271,12cyclictest0-21swapper/321:39:343
1341799271,12cyclictest0-21swapper/321:39:333
1339799272,18cyclictest0-21swapper/023:18:210
13397992715,4cyclictest0-21swapper/022:44:390
13397992715,3cyclictest0-21swapper/023:02:020
1339799271,20cyclictest0-21swapper/023:44:500
1341799269,10cyclictest0-21swapper/322:06:143
1341799261,12cyclictest0-21swapper/322:47:173
1341799261,12cyclictest0-21swapper/322:47:173
13417992610,3cyclictest24200-21pmu-power19:30:213
13417992610,3cyclictest0-21swapper/320:14:593
1341799260,12cyclictest0-21swapper/321:03:273
1341399266,9cyclictest25059-21dump-pmu-power19:35:012
13397992613,4cyclictest0-21swapper/023:27:420
1339799261,18cyclictest0-21swapper/022:16:350
1339799261,18cyclictest0-21swapper/021:42:220
1339799261,18cyclictest0-21swapper/000:00:560
1339799260,17cyclictest0-21swapper/021:46:200
59482250,1sleep00-21swapper/021:15:260
1341799259,8cyclictest0-21swapper/323:29:433
1341799259,3cyclictest0-21swapper/323:59:203
1341799259,3cyclictest0-21swapper/323:59:203
1341799259,3cyclictest0-21swapper/323:50:483
1341799259,3cyclictest0-21swapper/323:46:083
1341799259,3cyclictest0-21swapper/323:03:433
1341799259,3cyclictest0-21swapper/321:29:513
1341799259,3cyclictest0-21swapper/319:51:153
1341799256,4cyclictest4670-21cpuspeed_turbos21:15:133
13417992515,3cyclictest0-21swapper/322:03:223
13417992511,3cyclictest0-21swapper/320:21:353
1341799251,11cyclictest0-21swapper/321:47:123
13417992510,3cyclictest0-21swapper/320:52:353
13417992510,3cyclictest0-21swapper/319:17:353
13417992510,3cyclictest0-21swapper/319:17:353
13417992510,2cyclictest0-21swapper/319:44:553
1340599250,5cyclictest0-21swapper/122:05:111
1339799253,14cyclictest26770-21kworker/u8:023:30:300
13397992515,3cyclictest0-21swapper/022:56:200
13397992514,6cyclictest0-21swapper/023:53:510
1339799251,19cyclictest0-21swapper/021:59:580
1339799251,18cyclictest0-21swapper/022:04:180
1339799251,18cyclictest0-21swapper/021:12:320
1339799251,18cyclictest0-21swapper/000:20:020
1339799251,17cyclictest0-21swapper/021:34:330
1339799250,17cyclictest0-21swapper/023:38:040
1341799242,3cyclictest0-21swapper/300:35:173
13417992419,2cyclictest2111-21diskmemload22:27:333
1341799241,11cyclictest0-21swapper/300:01:043
1341799241,10cyclictest0-21swapper/319:11:113
13417992410,3cyclictest0-21swapper/322:56:493
1341399241,12cyclictest0-21swapper/223:00:382
1340599243,14cyclictest87650irq/129-enp1s0-23:50:011
1340599241,16cyclictest0-21swapper/100:10:581
1339799244,12cyclictest87350irq/126-enp1s0-00:05:170
1339799241,17cyclictest0-21swapper/000:28:510
1339799241,16cyclictest0-21swapper/019:41:120
1339799241,11cyclictest1484-21awk19:55:010
1339799240,17cyclictest9-21rcu_preempt23:45:160
1341799239,3cyclictest0-21swapper/321:34:513
1341799235,9cyclictest8299-21latency_hist20:10:023
1341799234,9cyclictest2608-21runrttasks22:23:333
1341799231,14cyclictest0-21swapper/321:40:263
13417992311,3cyclictest0-21swapper/321:51:063
1341799231,11cyclictest0-21swapper/322:16:333
1341799231,10cyclictest0-21swapper/323:21:233
1341799231,10cyclictest0-21swapper/321:10:033
1341399232,8cyclictest0-21swapper/223:50:152
1341399231,20cyclictest0-21swapper/223:47:332
1341399231,20cyclictest0-21swapper/222:03:272
1340599232,11cyclictest20337-21munin-run23:05:011
13405992317,3cyclictest19374-21grep20:35:131
1340599231,19cyclictest29073-21diskstats22:10:131
1340599231,19cyclictest24775-21ntp_states20:45:231
1340599230,12cyclictest18854-21kworker/u8:220:42:431
1339799238,9cyclictest0-21swapper/021:50:250
1339799237,9cyclictest0-21swapper/023:59:140
1339799237,9cyclictest0-21swapper/023:59:140
1339799231,15cyclictest0-21swapper/023:10:160
1339799231,13cyclictest0-21swapper/022:13:550
1339799231,12cyclictest22883-21perf19:30:020
1341799229,3cyclictest0-21swapper/323:36:033
1341799229,3cyclictest0-21swapper/321:59:323
1341799229,3cyclictest0-21swapper/300:24:433
1341799221,5cyclictest20741-21cstates23:05:133
1341799221,10cyclictest0-21swapper/320:44:113
13417992210,3cyclictest0-21swapper/300:34:313
1341399228,3cyclictest30-21rcuc/200:05:172
1341399224,6cyclictest1-21systemd19:10:102
1341399224,15cyclictest28425-21cut00:35:152
1341399221,6cyclictest13726-21munin-run22:50:012
1341399221,6cyclictest13726-21munin-run22:50:012
1341399221,4cyclictest23482-21cat21:55:252
1341399221,20cyclictest0-21swapper/200:14:222
1341399220,3cyclictest17924-21kworker/u8:123:19:252
1340599229,6cyclictest22001-21irqstats00:20:181
1340599226,3cyclictest18067-21df21:45:131
1340599222,4cyclictest15582-21munin-run19:15:021
1340599221,4cyclictest7818-21kworker/u8:021:31:121
1340599221,3cyclictest10310-21grep23:55:131
1340599221,3cyclictest10310-21grep23:55:131
1340599221,19cyclictest0-21swapper/123:27:311
1340599221,15cyclictest0-21swapper/122:35:261
1340599221,11cyclictest22249-21sort19:25:251
13405992210,11cyclictest18854-21kworker/u8:221:19:141
1340599220,8cyclictest0-21swapper/121:20:451
13397992218,3cyclictest0-21swapper/000:10:140
1339799221,16cyclictest0-21swapper/000:37:360
13397992211,3cyclictest11-21rcuc/020:05:170
1339799221,10cyclictest1500-21if_enp2s022:20:180
1339799221,10cyclictest14377-21ntpq21:35:210
1339799221,10cyclictest14377-21ntpq21:35:210
1339799221,10cyclictest0-21swapper/020:27:180
177572211,2sleep20-21swapper/222:55:302
1341799218,3cyclictest0-21swapper/322:52:093
1341799218,3cyclictest0-21swapper/300:14:483
1341799218,2cyclictest0-21swapper/322:36:313
1341799217,3cyclictest0-21swapper/322:42:093
1341799211,12cyclictest0-21swapper/323:15:133
1341799210,10cyclictest25051-21cat20:45:253
1341399216,3cyclictest2982-21smartctl19:55:232
1341399212,4cyclictest0-21swapper/221:51:392
1341399211,19cyclictest0-21swapper/222:52:132
1341399211,10cyclictest3412-21ntp_states21:10:212
1341399210,10cyclictest0-21swapper/221:16:272
1340599218,3cyclictest28704-21grep23:20:261
1340599216,4cyclictest0-21swapper/122:18:351
1340599214,2cyclictest87650irq/129-enp1s0-22:50:581
1340599212,3cyclictest0-21swapper/100:18:181
1340599211,8cyclictest0-21swapper/100:37:241
13405992117,3cyclictest0-21swapper/123:18:221
1340599211,18cyclictest0-21swapper/123:14:151
1340599211,17cyclictest4177-21/usr/sbin/munin22:25:231
1340599211,17cyclictest26574-21strings00:30:181
1340599211,11cyclictest5934-21cat20:05:001
1340599211,11cyclictest5505-21diskstats22:30:131
1340599211,11cyclictest29504-21cat19:45:011
1339799218,4cyclictest2111-21diskmemload00:30:130
1339799214,9cyclictest11-21rcuc/020:35:220
1339799212,11cyclictest18695-21/usr/sbin/munin19:20:240
1339799211,7cyclictest8322-21/usr/sbin/munin20:10:190
1339799211,15cyclictest87350irq/126-enp1s0-22:31:230
1339799211,15cyclictest0-21swapper/023:08:390
1339799211,11cyclictest1186-21ntp_states21:05:200
1339799211,10cyclictest29763-21cat19:45:120
1341799202,10cyclictest14918-21/usr/sbin/munin20:25:233
1341799201,8cyclictest0-21swapper/319:20:533
1341799201,12cyclictest839-21lldpd22:10:523
1341799201,12cyclictest0-21swapper/320:00:253
1341399205,3cyclictest24466-21if_enp2s020:45:172
1341399204,3cyclictest0-21swapper/223:35:212
1341399203,3cyclictest0-21swapper/223:26:162
1341399202,6cyclictest21190-21/usr/sbin/munin00:20:222
1341399201,7cyclictest15532-21/usr/sbin/munin21:40:242
1341399201,6cyclictest4023-21ntp_states22:25:212
1341399201,6cyclictest22425-21memory20:40:192
1341399201,6cyclictest19918-21ps19:20:212
1341399201,6cyclictest19899-21cat21:50:012
1341399201,5cyclictest29906-21processes22:10:222
1341399201,5cyclictest0-21swapper/223:13:412
13413992015,3cyclictest5889-21sh20:05:012
1341399201,10cyclictest0-21swapper/220:22:402
1340599208,3cyclictest22682-21cat23:10:011
1340599202,16cyclictest0-21swapper/121:59:491
13405992016,3cyclictest0-21swapper/119:22:211
1340599201,18cyclictest0-21swapper/121:02:011
1340599201,17cyclictest0-21swapper/122:45:231
1340599201,17cyclictest0-21swapper/122:45:231
1340599201,17cyclictest0-21swapper/120:32:491
1340599201,17cyclictest0-21swapper/100:00:021
1340599201,11cyclictest16472-21open_inodes19:15:041
1340599201,11cyclictest16472-21open_inodes19:15:041
1340599201,11cyclictest15913-21cat00:05:291
1340599201,10cyclictest24262-21ps19:30:211
1340599201,10cyclictest0-21swapper/123:40:111
13405992010,3cyclictest0-21swapper/122:23:241
1339799208,3cyclictest3996-21cpuspeed_turbos20:00:120
1339799201,7cyclictest12710-21/usr/sbin/munin20:20:190
1339799201,6cyclictest2960-21/usr/sbin/munin22:25:130
13397992016,2cyclictest18594-21sed20:30:240
13397992013,4cyclictest22124-21grep20:40:160
1341799196,3cyclictest0-21swapper/319:35:373
1341799191,3cyclictest0-21swapper/319:45:253
1341799191,13cyclictest7071-21df_inode21:20:113
1341799191,11cyclictest0-21swapper/319:57:173
1341399195,4cyclictest0-21swapper/222:40:172
1341399194,3cyclictest27242-21timerandwakeup20:50:242
1341399192,5cyclictest808-21snmpd22:19:432
1341399191,7cyclictest0-21swapper/223:30:552
1341399191,7cyclictest0-21swapper/223:05:142
1341399191,7cyclictest0-21swapper/200:26:042
1341399191,7cyclictest0-21swapper/200:03:512
1341399191,5cyclictest9726-21if_err_enp2s021:25:192
1341399191,11cyclictest0-21swapper/222:30:332
1341399191,10cyclictest0-21swapper/222:22:032
1341399191,10cyclictest0-21swapper/220:58:562
1340599199,2cyclictest6720-21gltestperf20:05:141
1340599197,3cyclictest30031-21diskstats19:45:141
1340599196,4cyclictest0-21swapper/122:43:351
1340599192,3cyclictest0-21swapper/123:39:441
1340599192,16cyclictest0-21swapper/119:58:101
1340599191,8cyclictest0-21swapper/100:28:351
13405991915,3cyclictest0-21swapper/121:51:101
13405991915,3cyclictest0-21swapper/121:25:141
13405991915,3cyclictest0-21swapper/121:10:441
1340599191,4cyclictest31545-21basename23:30:111
1340599191,2cyclictest87650irq/129-enp1s0-23:50:341
1340599191,17cyclictest0-21swapper/122:57:241
1340599191,10cyclictest10740-21cstates20:15:131
1339799195,3cyclictest15592-21idleruntime-cro19:15:010
1339799191,10cyclictest28098-21/usr/sbin/munin20:55:230
1341399185,3cyclictest87550irq/128-enp1s0-20:35:202
1341399183,6cyclictest917-21kworker/2:320:13:162
1341399183,4cyclictest18245-1kworker/u9:221:21:202
1341399181,7cyclictest0-21swapper/222:35:192
1341399181,5cyclictest13324-21sshd21:35:122
1341399181,5cyclictest13324-21sshd21:35:112
1341399181,5cyclictest10520-21/usr/sbin/munin20:15:242
1341399181,4cyclictest17115-21/usr/sbin/munin20:30:112
1341399181,4cyclictest14918-21/usr/sbin/munin20:25:232
1341399181,3cyclictest30361-21hddtemp_smartct19:45:162
1341399180,9cyclictest0-21swapper/221:07:392
1340599186,11cyclictest7818-21kworker/u8:020:20:411
1340599181,9cyclictest0-21swapper/122:00:071
13405991815,2cyclictest0-21swapper/120:55:381
13405991815,2cyclictest0-21swapper/119:35:051
1340599181,3cyclictest0-21swapper/119:53:481
1340599181,11cyclictest0-21swapper/120:10:221
1340599181,10cyclictest16295-21timerandwakeup20:25:261
1339799181,4cyclictest31359-21ntp_states21:00:210
1339799181,4cyclictest26798-21grep19:35:270
1339799181,4cyclictest15908-21smart_sda19:15:020
1339799181,4cyclictest15908-21smart_sda19:15:020
13397991813,3cyclictest808-21snmpd19:34:100
1341799170,10cyclictest0-21swapper/320:56:533
1341399173,3cyclictest87650irq/129-enp1s0-19:25:192
1341399171,6cyclictest0-21swapper/219:16:092
1341399171,6cyclictest0-21swapper/219:16:082
1341399171,5cyclictest0-21swapper/221:33:362
1341399171,4cyclictest6119-21/usr/sbin/munin20:05:182
1341399171,4cyclictest27137-21taskset23:20:102
1341399161,3cyclictest87350irq/126-enp1s0-19:35:262
1341399150,14cyclictest0-21swapper/221:02:372
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional