You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-30 - 09:33

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackcslot7s.osadl.org (updated Thu Jul 30, 2026 00:44:48)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
6210921290,3sleep20-21swapper/219:45:012
13386121080,3sleep20-21swapper/223:10:122
15732621020,3sleep20-21swapper/200:15:162
482132850,7sleep10-21swapper/119:05:211
4758928047,23sleep00-21swapper/019:05:130
4892326545,13sleep30-21swapper/319:08:383
4900426344,13sleep20-21swapper/219:09:402
1374152470,3sleep00-21swapper/023:20:120
1198502410,3sleep10-21swapper/122:30:011
4918299388,16cyclictest0-21swapper/223:42:592
1092972380,2sleep0109296-21apt-config22:00:000
4918299363,20cyclictest0-21swapper/220:36:172
4918299362,20cyclictest0-21swapper/223:52:382
4918299358,14cyclictest0-21swapper/219:28:252
4918299355,16cyclictest0-21swapper/219:51:312
4918299353,19cyclictest0-21swapper/220:58:192
4918299352,19cyclictest0-21swapper/222:56:532
4918299350,18cyclictest73113-21seq20:15:132
4918299344,17cyclictest0-21swapper/221:20:372
4918299343,17cyclictest0-21swapper/219:18:062
4918299332,16cyclictest0-21swapper/223:06:392
4918299332,16cyclictest0-21swapper/200:13:272
4918299331,17cyclictest0-21swapper/200:07:182
4918299330,18cyclictest131540-21chrt23:01:102
4918299330,18cyclictest131540-21chrt23:01:102
4918299330,17cyclictest140525-21munin-run23:30:002
4918299322,18cyclictest0-21swapper/223:37:192
4918299321,16cyclictest126653-21munin-run22:50:012
4918299320,20cyclictest137057-21turbostat.cron23:24:552
4918299320,18cyclictest50821-21taskset19:14:572
4918299320,17cyclictest98052-21python321:25:192
4918299313,16cyclictest0-21swapper/223:47:202
4918299310,18cyclictest89842-21chrt21:03:192
867172300,3sleep10-21swapper/120:55:011
4918299302,13cyclictest0-21swapper/221:40:162
4918299290,17cyclictest978-21snmpd20:12:132
4918299290,17cyclictest84461-21chrt20:45:562
4918299290,17cyclictest0-21swapper/221:51:002
49182992811,15cyclictest165337-21cron00:40:002
4918299272,12cyclictest0-21swapper/220:05:292
4918299272,12cyclictest0-21swapper/220:05:292
49177992719,6cyclictest883-21dbus-daemon23:00:331
49177992719,6cyclictest883-21dbus-daemon23:00:321
4918299260,14cyclictest0-21swapper/219:20:182
49172992611,12cyclictest141526-21ntp_states23:30:170
4918299259,14cyclictest110264-21cut22:00:162
49182992512,11cyclictest2445-21gdbus23:59:592
4918299248,14cyclictest82106-21grep20:40:192
49182992419,4cyclictest0-21swapper/221:45:092
49182992410,12cyclictest94387-21grep21:15:162
49177992415,7cyclictest436-20systemd-journal21:20:001
4918299239,12cyclictest127028-21cpuspeed_turbos22:50:122
4918299239,12cyclictest127028-21cpuspeed_turbos22:50:112
4918299237,14cyclictest161393-21grep00:25:212
49182992310,12cyclictest101197-21latency21:35:152
4918299230,1cyclictest0-21swapper/222:17:132
4918299230,1cyclictest0-21swapper/222:14:222
49172992314,7cyclictest1-21systemd00:04:590
1295912230,2sleep33056-21PK-Backend22:55:253
4918799220,20cyclictest3056-21PK-Backend22:48:403
4918799220,20cyclictest3056-21PK-Backend19:28:393
4918299229,12cyclictest0-21swapper/222:36:112
4918299224,16cyclictest162581-21if_enp2s000:30:172
49182992220,1cyclictest0-21swapper/222:44:372
49182992215,5cyclictest112772-21cron22:10:002
49182992215,5cyclictest112772-21cron22:10:002
4918299221,18cyclictest90999-21ntp_states21:05:212
4918299221,18cyclictest90999-21ntp_states21:05:212
4918299220,2cyclictest1002932chrt21:34:002
4918299220,20cyclictest118782-21sed22:25:162
4918299220,20cyclictest0-21swapper/219:30:302
49177992220,1cyclictest0-21swapper/122:05:491
49177992220,1cyclictest0-21swapper/122:05:481
49177992220,1cyclictest0-21swapper/121:21:071
49177992219,2cyclictest3056-21PK-Backend23:47:081
49177992218,1cyclictest0-21swapper/122:56:321
49177992216,4cyclictest142071-21taskset23:32:481
49177992213,6cyclictest111040-21cron22:05:001
49172992214,6cyclictest883-21dbus-daemon23:50:350
4917299220,21cyclictest0-21swapper/020:46:050
49187992114,5cyclictest436-20systemd-journal00:04:593
49187992113,6cyclictest89389-21swap21:00:213
4918799210,1cyclictest3056-21PK-Backend20:08:543
4918799210,1cyclictest3056-21PK-Backend20:08:543
4918799210,19cyclictest3056-21PK-Backend23:55:193
4918799210,19cyclictest3056-21PK-Backend23:11:583
4918799210,19cyclictest3056-21PK-Backend21:15:053
4918799210,19cyclictest3056-21PK-Backend20:32:203
4918799210,19cyclictest3056-21PK-Backend20:10:373
4918799210,19cyclictest3056-21PK-Backend19:30:213
4918299217,12cyclictest978-21snmpd21:55:472
49182992119,1cyclictest0-21swapper/223:32:432
49182992119,1cyclictest0-21swapper/223:15:592
49182992119,1cyclictest0-21swapper/220:30:122
49182992119,1cyclictest0-21swapper/220:20:322
49182992116,4cyclictest886-21NetworkManager22:33:012
49182992116,4cyclictest0-21swapper/219:56:192
4918299210,20cyclictest0-21swapper/219:45:172
4918299210,19cyclictest85057-21apt20:50:112
4917799219,9cyclictest883-21dbus-daemon20:15:001
49177992119,1cyclictest0-21swapper/123:54:441
49177992119,1cyclictest0-21swapper/123:20:451
49177992119,1cyclictest0-21swapper/119:38:541
49177992118,2cyclictest3056-21PK-Backend00:11:151
49177992118,2cyclictest0-21swapper/100:31:151
49177992117,2cyclictest0-21swapper/121:10:261
49177992115,4cyclictest117979-21sh22:25:001
49177992113,6cyclictest61263-21ntp_states19:40:191
49177992112,6cyclictest77840-21cron20:30:001
4917799210,20cyclictest0-21swapper/121:42:101
4917799210,20cyclictest0-21swapper/120:16:251
4917799210,20cyclictest0-21swapper/119:32:181
4917799210,20cyclictest0-21swapper/100:24:521
49172992119,1cyclictest0-21swapper/020:04:280
49172992119,1cyclictest0-21swapper/000:38:110
49172992116,3cyclictest129395-21ntp_states22:55:230
49172992114,6cyclictest70942-21pluginstate20:10:000
49172992114,6cyclictest70942-21pluginstate20:10:000
49172992112,7cyclictest97931-21ntp_states21:25:180
4917299210,20cyclictest0-21swapper/022:50:360
4917299210,20cyclictest0-21swapper/022:50:360
4917299210,20cyclictest0-21swapper/022:02:170
4917299210,20cyclictest0-21swapper/021:51:560
4917299210,20cyclictest0-21swapper/019:38:320
4917299210,1cyclictest80256-21smartctl20:35:170
49187992018,1cyclictest3056-21PK-Backend23:30:253
49187992018,1cyclictest3056-21PK-Backend22:17:453
49187992018,1cyclictest0-21swapper/319:10:343
49187992017,2cyclictest3056-21PK-Backend23:51:063
4918799200,1cyclictest3056-21PK-Backend21:35:133
4918799200,19cyclictest3056-21PK-Backend22:40:093
4918799200,19cyclictest3056-21PK-Backend22:14:543
4918799200,19cyclictest3056-21PK-Backend21:34:393
4918799200,19cyclictest3056-21PK-Backend19:41:303
4918799200,19cyclictest3056-21PK-Backend19:39:483
4918799200,19cyclictest3056-21PK-Backend19:22:033
4918799200,19cyclictest3056-21PK-Backend00:17:293
4918799200,18cyclictest3056-21PK-Backend23:27:543
4918799200,18cyclictest3056-21PK-Backend21:10:463
4918299203,16cyclictest0-21swapper/222:21:282
4918299203,15cyclictest0-21swapper/200:00:282
49182992018,1cyclictest0-21swapper/220:25:052
49182992018,1cyclictest0-21swapper/219:35:032
4918299201,17cyclictest91965-21/usr/sbin/munin21:10:252
4918299200,18cyclictest159026-21expr00:20:132
4917799209,7cyclictest1-21systemd21:55:321
49177992019,1cyclictest0-21swapper/120:09:151
49177992019,1cyclictest0-21swapper/120:09:151
49177992018,1cyclictest0-21swapper/123:36:271
49177992018,1cyclictest0-21swapper/123:17:071
49177992018,1cyclictest0-21swapper/121:29:121
49177992018,1cyclictest0-21swapper/120:38:291
49177992018,1cyclictest0-21swapper/120:03:481
49177992018,1cyclictest0-21swapper/119:49:281
49177992011,6cyclictest54863-21sshd19:22:421
49177992010,7cyclictest883-21dbus-daemon19:12:351
4917799200,1cyclictest0-21swapper/119:51:111
4917799200,19cyclictest0-21swapper/123:29:431
4917799200,19cyclictest0-21swapper/123:10:581
4917799200,19cyclictest0-21swapper/121:49:061
4917799200,19cyclictest0-21swapper/121:37:491
4917799200,19cyclictest0-21swapper/120:23:011
4917799200,19cyclictest0-21swapper/100:37:241
4917799200,19cyclictest0-21swapper/100:02:361
4917799200,18cyclictest3056-21PK-Backend00:05:061
49172992018,1cyclictest0-21swapper/021:12:350
49172992018,1cyclictest0-21swapper/000:31:150
49172992016,3cyclictest124916-21turbostat22:45:000
49172992014,5cyclictest883-21dbus-daemon20:28:030
49172992014,3cyclictest1895007-21snapd22:13:430
49172992013,5cyclictest883-21dbus-daemon23:20:000
49172992012,7cyclictest145751-21unin-run23:45:010
49172992012,6cyclictest53311-21cron19:20:000
4917299200,1cyclictest0-21swapper/022:30:290
4917299200,19cyclictest3056-21PK-Backend23:00:440
4917299200,19cyclictest3056-21PK-Backend23:00:440
4917299200,19cyclictest0-21swapper/023:56:530
4917299200,19cyclictest0-21swapper/023:45:590
4917299200,19cyclictest0-21swapper/022:16:350
4917299200,19cyclictest0-21swapper/021:30:060
4917299200,19cyclictest0-21swapper/020:53:280
4917299200,19cyclictest0-21swapper/020:40:580
4917299200,19cyclictest0-21swapper/019:55:020
4917299200,19cyclictest0-21swapper/019:26:500
4917299200,19cyclictest0-21swapper/019:11:220
49187991917,1cyclictest3056-21PK-Backend23:40:113
49187991917,1cyclictest3056-21PK-Backend20:20:373
49187991917,1cyclictest3056-21PK-Backend00:21:213
4918799191,6cyclictest52718-21sensors_temp19:15:223
49187991916,2cyclictest3056-21PK-Backend00:37:103
4918799190,1cyclictest3056-21PK-Backend23:18:423
4918799190,1cyclictest3056-21PK-Backend22:03:253
4918799190,1cyclictest3056-21PK-Backend21:45:343
4918799190,1cyclictest3056-21PK-Backend21:28:373
4918799190,1cyclictest3056-21PK-Backend21:06:273
4918799190,1cyclictest3056-21PK-Backend21:06:273
4918799190,1cyclictest3056-21PK-Backend00:30:123
4918799190,1cyclictest0-21swapper/320:18:213
4918799190,18cyclictest3056-21PK-Backend21:41:493
4918799190,18cyclictest0-21swapper/300:13:373
4918799190,17cyclictest88277-21perf-exec21:00:003
4918299192,15cyclictest67844-21cpuspeed_turbos20:00:122
4917799193,14cyclictest89976-21cron21:05:001
4917799192,12cyclictest90257-21apt21:05:131
4917799192,12cyclictest90257-21apt21:05:121
49177991917,1cyclictest0-21swapper/122:10:421
49177991917,1cyclictest0-21swapper/121:50:141
49177991917,1cyclictest0-21swapper/121:33:311
49177991917,1cyclictest0-21swapper/119:55:091
49177991913,5cyclictest151195-21/usr/sbin/munin00:00:001
49177991910,7cyclictest84797-21cron20:50:001
49177991910,7cyclictest78120-21apt20:30:121
49177991910,7cyclictest51403-21users19:15:011
49177991910,6cyclictest436-20systemd-journal22:16:061
4917799190,1cyclictest0-21swapper/100:19:521
49172991917,1cyclictest0-21swapper/023:36:120
49172991917,1cyclictest0-21swapper/023:12:130
49172991917,1cyclictest0-21swapper/023:09:420
49172991917,1cyclictest0-21swapper/021:36:070
49172991917,1cyclictest0-21swapper/000:20:270
49172991917,1cyclictest0-21swapper/000:06:550
49172991914,4cyclictest13-21ksoftirqd/021:15:180
49172991913,5cyclictest13-21ksoftirqd/019:45:100
49172991913,4cyclictest157634-21ntp_kernel_err00:15:190
49172991912,5cyclictest97072-21date21:25:010
4917299190,5cyclictest72648-21idleruntime-cro20:15:000
4917299190,1cyclictest0-21swapper/021:42:090
4917299190,1cyclictest0-21swapper/020:32:260
4917299190,18cyclictest0-21swapper/021:47:010
4917299190,18cyclictest0-21swapper/020:17:190
4917299190,18cyclictest0-21swapper/000:12:430
49187991816,1cyclictest3056-21PK-Backend22:05:073
49187991816,1cyclictest3056-21PK-Backend22:05:073
49187991816,1cyclictest3056-21PK-Backend21:58:313
49187991816,1cyclictest3056-21PK-Backend21:23:443
49187991816,1cyclictest3056-21PK-Backend20:35:103
49187991813,3cyclictest128385-21munin-run22:55:003
49187991813,3cyclictest128385-21munin-run22:55:003
49187991812,5cyclictest42-21ksoftirqd/323:00:153
49187991812,5cyclictest42-21ksoftirqd/323:00:153
49187991811,5cyclictest146414-21cat23:45:153
4917799189,7cyclictest436-20systemd-journal22:37:411
4917799187,8cyclictest1-21systemd22:50:011
4917799182,13cyclictest144254-21apt23:40:091
49177991813,3cyclictest161022-21missed_timers00:25:161
49177991811,5cyclictest123594-21grep22:40:131
49172991816,1cyclictest0-21swapper/019:41:570
49172991816,1cyclictest0-21swapper/000:26:480
49172991813,4cyclictest13-21ksoftirqd/022:40:010
49172991812,5cyclictest64669-21netstat19:50:190
49172991810,6cyclictest91031-21which21:05:210
49172991810,6cyclictest91031-21which21:05:210
49172991810,6cyclictest125566-21cat22:45:150
49172991810,6cyclictest117077-21cat22:20:180
49187991715,1cyclictest3056-21PK-Backend23:24:503
49187991715,1cyclictest3056-21PK-Backend00:27:433
49187991714,2cyclictest3056-21PK-Backend23:36:323
49187991714,2cyclictest3056-21PK-Backend20:01:433
49187991712,4cyclictest84201-21swap20:45:213
49187991712,3cyclictest85505-21if_enp2s020:50:163
49187991712,3cyclictest132601-21irqstats23:05:173
4918799170,15cyclictest153946-21ntp_states00:05:223
4917799179,5cyclictest57016-21latency_hist19:30:011
49177991715,1cyclictest0-21swapper/120:40:111
49177991712,3cyclictest86955-21cpuspeed_turbos20:55:111
49177991712,3cyclictest120033-21cpu22:30:121
4917299172,11cyclictest111319-21apt22:05:100
4917299172,11cyclictest111319-21apt22:05:100
49172991715,1cyclictest0-21swapper/023:27:340
49172991715,1cyclictest0-21swapper/019:21:220
4918799161,13cyclictest105823-21/usr/sbin/munin21:50:143
4917799167,3cyclictest0-21swapper/122:50:181
4917799167,3cyclictest0-21swapper/122:50:181
4917299169,5cyclictest57941-21wc19:30:200
49187991513,1cyclictest3056-21PK-Backend19:45:293
4918799150,13cyclictest3056-21PK-Backend22:25:203
4917799157,6cyclictest133595-21cron23:10:001
4917299158,5cyclictest436-20systemd-journal21:00:420
49172991513,1cyclictest0-21swapper/022:28:460
49172991513,1cyclictest0-21swapper/020:20:090
49187991410,3cyclictest3056-21PK-Backend20:29:523
4918799140,12cyclictest121284-21taskset22:33:043
49172991410,3cyclictest1095-21runrttasks20:57:390
661762130,4sleep33056-21PK-Backend19:55:123
49187991311,1cyclictest3056-21PK-Backend22:35:023
4918799130,12cyclictest3056-21PK-Backend20:42:523
4918799130,11cyclictest3056-21PK-Backend19:50:183
4918799128,3cyclictest3056-21PK-Backend22:20:453
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional