You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-16 - 11:30

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack2slot7s.osadl.org (updated Thu Apr 16, 2026 00:43:20)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1214968997249,20cyclictest1315431-21needreboot22:40:330
1214968997249,20cyclictest1315431-21needreboot22:40:330
1214969996954,10cyclictest420-21avahi-daemon20:50:421
1214970996541,3cyclictest171rcu_preempt19:10:352
1214968996336,3cyclictest0-21swapper/021:30:010
1214970995948,7cyclictest1-21systemd21:35:012
1214969995943,7cyclictest1056-21mta-sts-daemon21:25:371
1214968995829,8cyclictest0-21swapper/023:40:000
1214970995042,5cyclictest1240434-21cron20:35:012
1214970994939,4cyclictest0-21swapper/221:55:392
121497099492,39cyclictest1298445-21ssh22:15:382
1214970994536,5cyclictest1252704-21unixbench_multi21:10:492
1214970994536,5cyclictest1252704-21unixbench_multi21:10:492
1214969994536,6cyclictest1-21systemd21:00:011
1214968994520,19cyclictest0-21swapper/020:35:410
1214968994433,5cyclictest151ktimers/019:40:410
1214970994337,3cyclictest0-21swapper/200:00:012
1214970994333,6cyclictest1368112-21cups-browsed00:02:022
1214969994338,2cyclictest0-21swapper/122:25:341
1214969994333,6cyclictest1322546-21timerandwakeup22:50:451
1214969994333,6cyclictest1322546-21timerandwakeup22:50:451
1214969994333,6cyclictest0-21swapper/121:10:011
1214969994333,6cyclictest0-21swapper/100:15:011
1214969994332,7cyclictest3639778-21cups-browsed21:37:521
1214968994326,12cyclictest1332100-21needreboot23:05:340
1214968994322,14cyclictest3639778-21cups-browsed22:02:260
1214969994234,5cyclictest1331615-21aten_r2power_po23:05:271
121496999423,29cyclictest1017-21sendmail-mta22:15:441
121496999422,30cyclictest1294957-21ssh22:10:421
1214968994235,4cyclictest1056-21mta-sts-daemon19:10:420
1214970994135,2cyclictest0-21swapper/222:05:282
1214970994132,4cyclictest0-21swapper/222:00:362
1214969994131,7cyclictest1240436-21cron20:35:011
1214969994131,5cyclictest0-21swapper/121:55:391
1214969994130,5cyclictest0-21swapper/100:15:401
1214969994115,5cyclictest28-21ksoftirqd/122:30:401
1214968994131,8cyclictest1254662-21munin-plugin-st21:15:000
1214968994131,8cyclictest1254662-21munin-plugin-st21:15:000
1214968994127,11cyclictest1235938-21cron20:20:010
121496899412,33cyclictest1342378-21ssh23:20:410
121497099404,28cyclictest1328901-21ssh23:00:422
1214970994033,4cyclictest1235998-21latency_hist20:20:022
1214970994033,3cyclictest0-21swapper/222:20:252
1214970994032,4cyclictest0-21swapper/222:55:002
1214970994032,4cyclictest0-21swapper/222:55:002
1214970994031,5cyclictest0-21swapper/222:55:312
121496999403,26cyclictest1345583-21sh23:25:391
1214969994029,7cyclictest1252032-21needreboot21:10:341
1214969994029,7cyclictest1252032-21needreboot21:10:341
1214968994033,4cyclictest1311056-21sh22:35:010
1214968994028,9cyclictest1382598-21kernelversion00:20:330
1214970993929,4cyclictest0-21swapper/223:30:352
1214970993929,4cyclictest0-21swapper/223:30:352
1214970993928,8cyclictest1372537-21latency00:05:312
1214969993931,5cyclictest1375099-21latency_hist00:10:021
1214968993930,5cyclictest151ktimers/019:50:360
1214968993930,4cyclictest151ktimers/020:00:370
1214968993926,9cyclictest12350irq/32-eth023:10:400
1214968993924,9cyclictest1334557-21kworker/u8:1-xprtiod00:00:340
1214970993831,3cyclictest0-21swapper/200:10:362
1214970993828,6cyclictest1368112-21cups-browsed00:24:262
1214970993827,7cyclictest0-21swapper/222:30:382
1214970993826,9cyclictest1379443-21needreboot00:15:372
1214969993833,2cyclictest0-21swapper/123:40:001
1214969993829,5cyclictest1313504-21kworker/u8:0-xprtiod22:45:361
1214969993828,6cyclictest439-21systemd-logind00:35:011
1214968993831,4cyclictest3639778-21cups-browsed21:35:400
1214968993829,6cyclictest1327710-21idleruntime-cro23:00:000
1214968993828,6cyclictest1259316-21needreboot21:20:350
1214968993827,7cyclictest1279370-21ls21:50:020
1214968993826,8cyclictest1379282-21latency00:15:350
1214970993732,2cyclictest0-21swapper/222:40:012
1214970993729,5cyclictest3639778-21cups-browsed23:40:392
1214970993729,5cyclictest1392166-21munin-plugin-st00:35:012
1214970993729,5cyclictest0-21swapper/221:00:002
121497099371,6cyclictest171rcu_preempt20:00:382
121497099371,29cyclictest659-21snmpd20:20:392
1214969993730,3cyclictest0-21swapper/100:05:011
1214969993729,5cyclictest1055-21mta-sts-daemon21:32:331
1214969993726,7cyclictest420-21avahi-daemon23:13:081
1214969993725,9cyclictest1366320-21unixbench_multi23:55:491
121496899376,17cyclictest12350irq/32-eth022:25:380
1214968993732,3cyclictest1395673-21plymouth00:40:010
1214968993728,6cyclictest1284281-21sendmail_mailst21:55:460
1214968993714,19cyclictest1328462-21gltestperf23:00:320
121497099363,28cyclictest1359070-21ssh23:45:392
1214970993629,4cyclictest1352239-21ntp_kernel_err23:35:332
1214970993629,4cyclictest0-21swapper/219:20:012
1214970993624,8cyclictest1262085-21acpi21:25:252
1214970993620,14cyclictest0-21swapper/222:10:402
121496999369,20cyclictest1338965-21ssh23:15:381
121496999369,20cyclictest1338965-21ssh23:15:381
1214969993626,7cyclictest420-21avahi-daemon21:46:071
1214969993625,8cyclictest1327720-21sendmail23:00:001
121496999361,28cyclictest1302020-21ssh22:20:371
1214968993626,8cyclictest1291195-21missed_timers22:05:330
121496899362,26cyclictest1294996-21ssh22:10:440
1214968993621,9cyclictest12350irq/32-eth022:20:380
1214968993620,13cyclictest151ktimers/020:05:440
1214970993531,2cyclictest0-21swapper/221:25:002
1214970993527,5cyclictest1395662-21sendmail00:40:012
1214970993527,5cyclictest1218502-21fschecks_time19:20:322
1214970993527,5cyclictest1218502-21fschecks_time19:20:322
1214969993527,5cyclictest1255985-21proc_pri21:15:441
1214969993526,6cyclictest1329361-21unixbench-2d23:00:511
1214969993526,5cyclictest1234349-21unixbench_multi20:10:481
1214969993524,8cyclictest1226332-21ntp_kernel_pll_19:45:371
1214969993523,7cyclictest0-21swapper/122:00:381
1214969993512,6cyclictest28-21ksoftirqd/123:50:291
1214968993529,4cyclictest1349464-21unixbench_multi23:30:480
1214968993529,4cyclictest1349464-21unixbench_multi23:30:480
1214968993526,5cyclictest151ktimers/000:25:370
1214968993525,8cyclictest1346713-21ssh23:27:040
1214968993518,14cyclictest12350irq/32-eth020:40:420
1214970993428,3cyclictest0-21swapper/222:25:342
1214970993427,4cyclictest1228472-21systemd-run19:55:022
1214970993425,6cyclictest548-21gdbus19:55:462
121496999349,9cyclictest28-21ksoftirqd/119:10:001
1214969993427,4cyclictest0-21swapper/122:40:011
1214969993411,4cyclictest28-21ksoftirqd/122:05:051
1214968993426,5cyclictest1312567-21unixbench-2d22:35:470
1214968993425,6cyclictest1364996-21/usr/sbin/munin23:55:340
1214968993425,4cyclictest0-21swapper/021:30:390
1214968993424,7cyclictest261-21systemd-journal23:44:580
121496899342,24cyclictest1376253-21ssh00:10:390
1214968993421,10cyclictest0-21swapper/000:30:510
121497099334,26cyclictest3639778-21cups-browsed23:53:302
1214970993326,4cyclictest1344399-21systemd-run23:25:012
1214970993326,4cyclictest1241942-21munin-run20:40:012
1214970993323,7cyclictest600-21gmain23:17:102
1214970993323,7cyclictest600-21gmain23:17:102
1214969993325,5cyclictest1242031-21latency_hist20:40:021
121496999332,22cyclictest1348861-21ssh23:30:361
121496999332,22cyclictest1348861-21ssh23:30:361
1214969993312,5cyclictest28-21ksoftirqd/123:23:131
121497099323,23cyclictest1269728-21ssh21:35:392
1214970993224,3cyclictest0-21swapper/223:25:422
1214969993226,3cyclictest0-21swapper/120:45:011
1214969993225,4cyclictest0-21swapper/120:20:011
1214969993224,5cyclictest1315036-21cpuspeed22:40:261
1214969993224,5cyclictest1315036-21cpuspeed22:40:261
1214969993223,6cyclictest1259938-21unixbench-2d21:20:491
1214968993225,4cyclictest1234461-21systemd-run20:15:010
1214968993222,7cyclictest1220961-21munin-plugin-st19:30:010
1214968993220,10cyclictest420-21avahi-daemon21:40:480
1214968993211,6cyclictest0-21swapper/022:45:400
1214970993125,3cyclictest0-21swapper/222:40:412
1214970993125,3cyclictest0-21swapper/222:40:412
1214970993125,3cyclictest0-21swapper/200:25:272
1214970993124,5cyclictest0-21swapper/221:50:012
1214970993123,5cyclictest420-21avahi-daemon21:40:482
1214970993123,5cyclictest1258246-21sendmail21:20:002
121497099312,23cyclictest1280305-21ssh21:50:332
121496999314,17cyclictest1359070-21ssh23:45:391
1214969993125,3cyclictest1334557-21kworker/u8:1-xprtiod00:20:501
1214969993123,5cyclictest1232955-21munin-run20:10:011
1214969993122,5cyclictest1347818-21kworker/u8:9-rpciod23:40:281
1214969993122,5cyclictest1279678-21aten_r2power_en21:50:241
1214969993121,7cyclictest1229749-21timerwakeupswit19:55:501
1214969993112,5cyclictest28-21ksoftirqd/100:25:401
1214969993111,4cyclictest28-21ksoftirqd/121:40:341
1214968993125,4cyclictest1339483-21unixbench_multi23:15:480
1214968993125,4cyclictest1339483-21unixbench_multi23:15:480
1214968993123,4cyclictest0-21swapper/021:15:330
121496899311,25cyclictest1280483-21ssh21:50:400
1214970993023,3cyclictest0-21swapper/223:05:362
1214970993020,7cyclictest1055-21mta-sts-daemon20:15:022
121496999309,4cyclictest28-21ksoftirqd/119:25:381
1214969993023,4cyclictest1056-21mta-sts-daemon19:50:301
121496899303,25cyclictest0-21swapper/019:35:360
1214968993024,4cyclictest0-21swapper/020:55:390
1214968993023,4cyclictest1229885-21unixbench_singl19:55:510
1214968993021,6cyclictest420-21avahi-daemon19:45:430
1214968993017,7cyclictest1362258-21awk23:50:340
1214970992924,2cyclictest0-21swapper/223:15:022
1214970992922,4cyclictest0-21swapper/219:40:012
121496999299,4cyclictest28-21ksoftirqd/119:40:431
1214969992916,8cyclictest1236881-21ntpq20:20:381
1214969992911,4cyclictest28-21ksoftirqd/119:15:401
1214969992910,4cyclictest28-21ksoftirqd/120:25:331
1214969992910,4cyclictest28-21ksoftirqd/119:35:401
1214969992910,4cyclictest28-21ksoftirqd/119:20:291
1214969992910,4cyclictest28-21ksoftirqd/119:20:291
1214968992921,6cyclictest1218193-21sshd19:20:080
1214968992921,6cyclictest1218193-21sshd19:20:080
1214968992921,5cyclictest1246314-21tail20:50:470
1214968992916,10cyclictest0-21swapper/021:00:470
121496899291,23cyclictest1238955-21cut20:30:010
121497099284,3cyclictest171rcu_preempt19:30:422
1214970992822,3cyclictest1230089-21kworker/u8:8-xprtiod20:45:342
1214970992822,3cyclictest1230089-21kworker/u8:8-xprtiod20:45:342
1214970992822,3cyclictest0-21swapper/222:45:392
1214970992820,5cyclictest518-21rs:main2
121497099281,22cyclictest659-21snmpd20:50:352
121496999289,6cyclictest28-21ksoftirqd/119:15:021
121496999282,20cyclictest1368112-21cups-browsed00:35:371
1214969992821,4cyclictest1222383-21unixbench_singl19:30:471
1214968992822,4cyclictest1250150-21latency21:05:330
1214968992822,4cyclictest1221479-21fschecks_count19:30:290
1214968992821,4cyclictest624-21lldpd22:15:310
1214968992813,12cyclictest695-21cron19:20:010
1214970992723,2cyclictest0-21swapper/219:45:382
121496999279,6cyclictest28-21ksoftirqd/120:45:351
121496999279,6cyclictest28-21ksoftirqd/120:45:351
1214969992720,4cyclictest1249443-21cron21:05:001
1214969992710,4cyclictest28-21ksoftirqd/120:00:281
1214968992721,4cyclictest1358294-21/usr/sbin/munin23:45:390
121496899271,21cyclictest1372440-21awk00:05:300
1214970992617,6cyclictest440-21thd20:25:352
121496899261,14cyclictest1321403-21sort22:50:240
121496899261,14cyclictest1321403-21sort22:50:240
1214970992520,2cyclictest0-21swapper/221:10:022
1214970992517,5cyclictest1249346-21unixbench_multi21:00:482
121497099251,19cyclictest1220370-21sendmail-mta19:25:392
1214968992520,2cyclictest0-21swapper/020:50:010
1214968992520,2cyclictest0-21swapper/020:50:010
1214968992513,9cyclictest0-21swapper/020:30:470
1214968992417,4cyclictest0-21swapper/020:20:370
1214970992318,2cyclictest0-21swapper/219:45:012
1214968992314,6cyclictest1214982-21systemd19:10:010
1214970992215,5cyclictest1214974-21true19:10:012
1214970992015,3cyclictest1054-21mta-sts-daemon20:05:312
121497199181,9cyclictest0-21swapper/323:55:353
121497199172,9cyclictest0-21swapper/321:20:363
121497199172,9cyclictest0-21swapper/300:05:383
121497199171,8cyclictest0-21swapper/323:15:403
121497199171,8cyclictest0-21swapper/323:15:403
121497199163,7cyclictest0-21swapper/322:35:413
121497199162,9cyclictest0-21swapper/300:10:413
121497199162,8cyclictest0-21swapper/321:40:353
121497199162,8cyclictest0-21swapper/321:25:363
121497199162,7cyclictest0-21swapper/323:05:373
121497199162,7cyclictest0-21swapper/300:28:183
121497199162,7cyclictest0-21swapper/300:20:373
121497199155,8cyclictest0-21swapper/321:10:373
121497199155,8cyclictest0-21swapper/321:10:373
121497199153,8cyclictest0-21swapper/300:15:423
121497199153,7cyclictest0-21swapper/323:40:423
121497199153,6cyclictest0-21swapper/322:25:413
121497199152,8cyclictest0-21swapper/323:52:263
121497199152,8cyclictest0-21swapper/322:20:343
121497199152,8cyclictest0-21swapper/300:35:423
121497199152,7cyclictest0-21swapper/322:00:363
121497199151,8cyclictest0-21swapper/323:30:383
121497199151,8cyclictest0-21swapper/323:30:383
121497199151,8cyclictest0-21swapper/322:50:373
121497199151,8cyclictest0-21swapper/322:50:373
121497199151,6cyclictest0-21swapper/321:35:343
121497199151,6cyclictest0-21swapper/319:35:353
121497199151,6cyclictest0-21swapper/319:30:363
121497199145,6cyclictest0-21swapper/323:00:363
121497199143,6cyclictest0-21swapper/322:10:463
121497199143,6cyclictest0-21swapper/319:20:003
121497199142,8cyclictest0-21swapper/322:15:393
121497199142,8cyclictest0-21swapper/321:15:383
121497199142,7cyclictest0-21swapper/322:40:343
121497199142,7cyclictest0-21swapper/322:40:343
121497199142,6cyclictest0-21swapper/323:45:263
121497199142,5cyclictest0-21swapper/320:45:343
121497199142,5cyclictest0-21swapper/320:45:343
121497199141,8cyclictest0-21swapper/322:05:273
121497199141,8cyclictest0-21swapper/321:50:353
121497199141,8cyclictest0-21swapper/300:30:453
121497199141,7cyclictest0-21swapper/322:55:353
121497199141,7cyclictest0-21swapper/321:30:373
121497199141,6cyclictest0-21swapper/321:55:283
121497199141,6cyclictest0-21swapper/319:55:393
121497199140,7cyclictest0-21swapper/320:20:393
121497199133,6cyclictest0-21swapper/323:10:363
121497199133,6cyclictest0-21swapper/322:30:383
121497199133,6cyclictest0-21swapper/300:00:393
121497199132,7cyclictest0-21swapper/321:45:423
121497199132,6cyclictest0-21swapper/320:50:423
121497199132,5cyclictest0-21swapper/320:00:373
121497199131,8cyclictest0-21swapper/322:45:413
121497199131,7cyclictest0-21swapper/323:25:383
121497199131,7cyclictest0-21swapper/323:20:383
121497199131,6cyclictest0-21swapper/319:40:373
121497199131,6cyclictest0-21swapper/319:25:373
121497199131,6cyclictest0-21swapper/319:10:363
121497199131,5cyclictest0-21swapper/320:15:353
121497199131,5cyclictest0-21swapper/319:45:363
121497199122,6cyclictest0-21swapper/323:35:343
121497199122,6cyclictest0-21swapper/320:35:413
121497199122,5cyclictest0-21swapper/319:25:003
121497199122,5cyclictest0-21swapper/319:25:003
121497199121,6cyclictest0-21swapper/320:30:013
121497199121,6cyclictest0-21swapper/320:05:413
121497199112,6cyclictest0-21swapper/320:35:013
121497199112,5cyclictest0-21swapper/321:00:373
121497199111,6cyclictest0-21swapper/320:10:363
121497199102,5cyclictest0-21swapper/320:40:293
121497199101,6cyclictest0-21swapper/321:10:013
121497199101,6cyclictest0-21swapper/319:50:343
121497199101,4cyclictest0-21swapper/319:10:023
12149719992,4cyclictest0-21swapper/320:55:363
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional