You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-12 - 07:24

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack2slot7s.osadl.org (updated Sun Apr 12, 2026 00:43:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1325229997430,7cyclictest171rcu_preempt20:25:350
1325229996329,30cyclictest1427635-21rm22:30:360
1325231996151,7cyclictest1382852-21sh21:35:392
1325229995645,5cyclictest0-21swapper/023:25:420
1325231995038,8cyclictest3559463-21cups-browsed21:55:502
1325231995038,8cyclictest3559463-21cups-browsed21:55:502
1325231995017,6cyclictest171rcu_preempt22:05:402
1325230994942,4cyclictest1056-21mta-sts-daemon23:00:231
1325230994938,8cyclictest420-21avahi-daemon23:10:391
1325229994933,10cyclictest151ktimers/019:30:380
1325229994931,10cyclictest1419543-21sh22:20:340
1325229994840,6cyclictest12350irq/32-eth021:20:390
1325229994840,4cyclictest0-21swapper/023:48:270
1325229994828,3cyclictest171rcu_preempt19:40:330
1325231994737,6cyclictest0-21swapper/222:45:002
1325229994738,5cyclictest0-21swapper/022:35:380
1325230994614,7cyclictest171rcu_preempt21:35:371
1325231994531,10cyclictest261-21systemd-journal21:33:062
132523199451,37cyclictest1415484-21ssh22:15:362
1325229994536,5cyclictest0-21swapper/023:22:130
1325229994536,5cyclictest0-21swapper/023:22:130
1325229994530,11cyclictest12350irq/32-eth021:45:400
1325229994515,21cyclictest1476463-21ssh23:30:430
132523099443,36cyclictest1370603-21ssh21:20:381
132523099441,5cyclictest171rcu_preempt22:05:391
1325229994428,10cyclictest1389883-21latency_hist21:45:020
1325231994337,3cyclictest0-21swapper/221:48:082
1325231994334,5cyclictest261-21systemd-journal23:25:012
1325231994334,5cyclictest261-21systemd-journal23:25:012
1325230994336,3cyclictest0-21swapper/100:20:331
1325230994234,5cyclictest1341743-21systemd-run20:05:011
1325230994232,6cyclictest1515612-21cut00:20:011
1325229994236,3cyclictest0-21swapper/020:35:010
1325229994228,11cyclictest1503975-21aten_r2power_en00:05:240
132522999422,33cyclictest1366611-21sh21:15:420
1325231994133,5cyclictest1487260-21systemd-run23:45:012
1325231994133,5cyclictest1329743-21systemd-run19:25:012
1325231994132,6cyclictest1-21systemd23:15:442
1325231994132,6cyclictest1-21systemd23:15:442
1325231994131,7cyclictest1337229-21sh19:50:012
1325229994134,4cyclictest1456195-21sh23:05:430
132523199404,32cyclictest1387059-21ssh21:40:422
1325231994032,5cyclictest261-21systemd-journal00:05:012
1325231994032,5cyclictest1356722-21systemd-run20:55:022
1325230994032,5cyclictest3559463-21cups-browsed22:10:191
1325230994032,5cyclictest1358237-21systemd-run21:00:021
1325230994031,6cyclictest1326734-21systemd-run19:15:011
1325229994018,17cyclictest151ktimers/019:15:380
132522999401,36cyclictest0-21swapper/019:25:400
132522999401,36cyclictest0-21swapper/019:25:400
132523099393,31cyclictest1508729-21ssh00:10:361
1325230993931,5cyclictest1503681-21latency_hist00:05:021
1325230993930,6cyclictest261-21systemd-journal22:55:001
1325230993926,10cyclictest271ktimers/120:15:401
1325229993932,3cyclictest0-21swapper/000:20:010
1325229993931,4cyclictest0-21swapper/021:35:280
1325231993831,4cyclictest1459125-21kworker/u8:14-xprtiod00:25:012
1325231993830,5cyclictest1531818-21munin-run00:40:012
1325231993830,5cyclictest1531818-21munin-run00:40:012
1325231993829,7cyclictest1373454-21sh21:25:012
1325229993831,4cyclictest1418359-21munin-run22:20:010
1325229993829,5cyclictest1495490-21latency_hist23:55:020
132522999382,30cyclictest1411469-21ssh22:10:370
1325231993731,4cyclictest1483173-21awk23:40:002
1325231993731,3cyclictest0-21swapper/200:15:422
1325231993730,4cyclictest0-21swapper/223:05:012
1325231993729,5cyclictest1411856-21awk22:10:482
1325231993728,6cyclictest1475122-21systemd-run23:30:012
1325231993728,5cyclictest1365284-21sh21:15:012
1325231993727,7cyclictest1369358-21run-parts21:20:012
132523199372,25cyclictest1443892-21ssh22:50:362
1325231993721,5cyclictest1450842-21sendmail23:00:012
132523199371,30cyclictest1423641-21ssh22:25:352
1325230993730,4cyclictest1450858-21kworker/u8:4-xprtiod23:05:421
1325230993729,5cyclictest1393906-21munin-plugin-st21:50:011
132523099372,27cyclictest1431834-21ssh22:35:401
1325229993730,3cyclictest0-21swapper/021:55:430
1325229993730,3cyclictest0-21swapper/021:55:430
1325229993727,4cyclictest0-21swapper/000:35:420
1325229993727,4cyclictest0-21swapper/000:35:420
1325229993725,10cyclictest0-21swapper/021:10:300
1325229993722,12cyclictest1487238-21cron23:45:010
1325231993628,5cyclictest421-21dbus-daemon23:13:022
1325231993627,6cyclictest1442730-21unin-run22:50:012
132523099363,25cyclictest1386843-21ssh21:40:341
1325230993630,3cyclictest0-21swapper/100:00:011
1325230993628,5cyclictest1425477-21sshd22:28:061
1325230993627,6cyclictest1403553-21systemctl22:00:411
1325230993627,6cyclictest1354229-21fschecks_count20:45:301
1325230993627,6cyclictest1340192-21cron20:00:011
132522999365,20cyclictest1395117-21ssh21:50:370
1325229993628,6cyclictest1346233-21sendmail20:20:010
1325231993528,5cyclictest341rcuc/219:10:382
1325231993526,6cyclictest1492042-21fschecks_count23:50:292
132523099353,25cyclictest1378676-21diskmemload21:30:371
1325230993528,4cyclictest1055-21mta-sts-daemon00:25:101
1325230993526,6cyclictest420-21avahi-daemon20:50:421
1325230993526,6cyclictest1499375-21cups-browsed00:05:521
1325230993526,6cyclictest1436419-21unixbench_singl22:40:471
1325230993523,5cyclictest1051-21mta-sts-daemon19:50:021
132523099352,28cyclictest1447921-21ssh22:55:351
132522999354,27cyclictest1480249-21rm23:35:330
1325229993527,5cyclictest1499375-21cups-browsed00:30:380
1325231993429,2cyclictest0-21swapper/221:10:012
1325231993426,5cyclictest1056-21mta-sts-daemon23:05:352
1325231993426,4cyclictest0-21swapper/219:50:412
1325231993423,7cyclictest1395093-21sh21:50:362
1325230993427,3cyclictest0-21swapper/123:15:361
1325230993427,3cyclictest0-21swapper/123:15:361
1325230993426,5cyclictest1484375-21ntp_kernel_pll_23:40:361
1325230993426,5cyclictest1418359-21munin-run22:20:011
1325230993425,6cyclictest1442823-21latency_hist22:50:021
132522999342,24cyclictest1520857-21ssh00:25:360
1325231993326,4cyclictest1347711-21cron20:25:012
1325231993325,5cyclictest1359710-21cron21:05:012
1325231993325,5cyclictest1-21systemd00:35:022
1325231993324,6cyclictest421-21dbus-daemon22:40:012
1325231993318,11cyclictest421-21dbus-daemon22:05:012
1325230993327,4cyclictest1325228-21cyclictest23:26:321
1325230993325,5cyclictest1524239-21sh00:30:271
1325230993325,5cyclictest1-21systemd23:55:021
1325230993321,5cyclictest1377525-21ssh21:30:011
1325229993326,4cyclictest1056-21mta-sts-daemon22:09:050
1325229993324,6cyclictest0-21swapper/022:45:480
1325229993324,5cyclictest1496801-21mailstats23:55:460
1325229993321,10cyclictest0-21swapper/000:05:010
1325231993226,3cyclictest0-21swapper/200:00:012
1325231993223,6cyclictest1488368-21ntp_kernel_err23:45:332
1325231993222,7cyclictest428-21mta-sts-daemon20:35:362
1325231993222,3cyclictest1352579-21aten_r2power_po20:40:252
1325231993222,3cyclictest1352579-21aten_r2power_po20:40:252
132523199322,21cyclictest1476369-21ssh23:30:402
132523199321,7cyclictest171rcu_preempt22:20:342
132523099323,21cyclictest1480348-21ssh23:35:361
1325230993225,4cyclictest1054-21mta-sts-daemon23:49:201
1325230993224,5cyclictest1427106-21aten_r2power_vo22:30:271
1325230993224,5cyclictest1359734-21systemd-run21:05:011
1325230993223,5cyclictest1352254-21latency_hist20:40:011
1325229993226,4cyclictest1460421-21switchtime23:10:450
1325229993224,5cyclictest1353737-21systemd-run20:45:010
1325229993224,5cyclictest1353737-21systemd-run20:45:010
132522999322,25cyclictest1464247-21ssh23:15:400
132522999322,25cyclictest1464247-21ssh23:15:400
132522999322,25cyclictest1378632-21ssh21:30:350
1325229993221,7cyclictest1499375-21cups-browsed00:10:050
1325229993215,14cyclictest1403455-21diskmemload22:00:400
132523199313,22cyclictest1374593-21ssh21:25:352
1325231993124,4cyclictest1054-21mta-sts-daemon22:32:352
1325231993123,5cyclictest1507929-21/usr/sbin/munin00:10:362
1325231993123,5cyclictest1343237-21unin-run20:10:012
132523099316,3cyclictest171rcu_preempt23:30:411
1325230993126,2cyclictest0-21swapper/121:10:301
132523099312,23cyclictest1399135-21ssh21:55:341
132523099312,23cyclictest1399135-21ssh21:55:341
1325230993120,8cyclictest1468540-21switchtime23:20:451
1325230993120,8cyclictest1468540-21switchtime23:20:451
132522999318,4cyclictest171rcu_preempt20:20:350
1325229993126,2cyclictest0-21swapper/020:40:010
1325229993119,9cyclictest1375271-21unixbench_singl21:25:480
1325229993118,10cyclictest1454859-21ssh23:05:010
132523199302,21cyclictest1504628-21ssh00:05:342
1325230993024,3cyclictest0-21swapper/122:25:011
1325230993023,4cyclictest0-21swapper/120:25:011
1325230993018,3cyclictest171rcu_preempt00:35:251
1325230993018,3cyclictest171rcu_preempt00:35:251
1325229993014,7cyclictest1447955-21ntpq22:55:360
1325231992923,3cyclictest1051-21mta-sts-daemon00:29:432
132523099292,21cyclictest1366515-21ssh21:15:381
1325229992921,5cyclictest0-21swapper/020:05:350
1325229992921,4cyclictest0-21swapper/022:41:520
1325231992821,4cyclictest1055-21mta-sts-daemon19:10:012
1325231992819,5cyclictest1331240-21systemd-run19:30:012
1325231992819,5cyclictest1331240-21systemd-run19:30:012
1325230992822,3cyclictest1320042-21kworker/u8:4-nfsiod19:30:011
1325230992822,3cyclictest1320042-21kworker/u8:4-nfsiod19:30:011
1325230992822,3cyclictest0-21swapper/119:40:011
1325230992820,5cyclictest3559463-21cups-browsed21:52:101
1325230992820,5cyclictest1334994-21missed_timers19:40:301
1325229992822,3cyclictest0-21swapper/022:50:290
132522999282,22cyclictest1423599-21ssh22:25:330
1325229992821,4cyclictest1329303-21systemctl19:20:390
1325229992819,7cyclictest1516517-21kernelversion00:20:290
1325229992817,8cyclictest0-21swapper/019:15:000
1325229992814,4cyclictest1341737-21systemd-run20:05:010
132523199277,14cyclictest428-21mta-sts-daemon20:30:372
1325231992720,4cyclictest1334256-21systemd19:40:022
1325231992719,5cyclictest1346078-21tail20:15:502
1325230992717,7cyclictest1327579-21ntp_kernel_pll_19:15:331
132523199265,7cyclictest0-21swapper/220:00:352
1325231992621,2cyclictest0-21swapper/220:00:012
1325231992618,5cyclictest1358108-21unixbench_singl20:55:482
1325231992611,8cyclictest0-21swapper/219:15:322
1325230992617,6cyclictest1346253-21kworker/u8:13+xprtiod21:10:011
1325230992613,8cyclictest171rcu_preempt20:10:261
132523199259,3cyclictest1332070-21ntpq19:30:322
1325231992518,4cyclictest1056-21mta-sts-daemon20:46:072
1325229992520,3cyclictest151ktimers/020:55:380
1325229992516,7cyclictest1054-21mta-sts-daemon20:10:460
1325229992515,7cyclictest3559463-21cups-browsed19:55:400
132522999251,19cyclictest1354766-21ps20:45:430
1325231992417,4cyclictest1349114-21unixbench_multi20:25:472
1325231992416,5cyclictest1335648-21unixbench_singl19:40:452
132523199240,8cyclictest1343199-21kworker/2:1+events_freezable20:10:342
1325230992418,4cyclictest1325228-21cyclictest20:25:291
1325230992418,3cyclictest1056-21mta-sts-daemon20:40:391
1325230992418,3cyclictest1056-21mta-sts-daemon20:40:391
132522999245,12cyclictest1054-21mta-sts-daemon19:35:350
1325229992417,4cyclictest1361213-21grep21:10:010
132523099236,8cyclictest171rcu_preempt20:30:351
1325230992317,3cyclictest0-21swapper/120:05:351
132522999239,3cyclictest0-21swapper/019:45:330
1325229992312,9cyclictest0-21swapper/019:50:280
132523099222,15cyclictest1329076-21ntpq19:20:311
1325230992216,3cyclictest1054-21mta-sts-daemon19:31:201
132522999229,10cyclictest0-21swapper/020:50:410
132522999222,15cyclictest1359141-21ntpq21:00:370
132523099219,9cyclictest261rcuc/119:10:021
1325230992013,5cyclictest516-21in:imuxsock19:50:031
132522999191,6cyclictest1325408-21latency_hist19:10:020
132523299182,10cyclictest0-21swapper/321:45:013
132523299181,8cyclictest0-21swapper/321:20:363
132523299174,6cyclictest0-21swapper/321:10:363
132523299173,7cyclictest0-21swapper/300:05:333
132523299163,8cyclictest0-21swapper/323:25:423
132523299163,8cyclictest0-21swapper/323:20:343
132523299163,8cyclictest0-21swapper/323:20:343
132523299163,7cyclictest0-21swapper/321:30:393
132523299162,9cyclictest0-21swapper/323:30:463
132523299161,9cyclictest0-21swapper/322:10:443
132523299161,8cyclictest0-21swapper/322:25:383
132523299161,8cyclictest0-21swapper/320:15:353
132523299161,7cyclictest0-21swapper/320:00:333
132523299152,9cyclictest0-21swapper/323:35:363
132523299152,8cyclictest0-21swapper/323:45:333
132523299152,8cyclictest0-21swapper/323:40:353
132523299152,8cyclictest0-21swapper/321:50:363
132523299152,8cyclictest0-21swapper/321:35:353
132523299152,6cyclictest0-21swapper/321:00:333
132523299151,8cyclictest0-21swapper/323:00:383
132523299151,8cyclictest0-21swapper/322:50:343
132523299151,8cyclictest0-21swapper/320:20:383
132523299151,8cyclictest0-21swapper/300:15:353
132523299151,7cyclictest0-21swapper/321:25:253
132523299151,7cyclictest0-21swapper/300:35:383
132523299151,7cyclictest0-21swapper/300:35:383
132523299151,6cyclictest0-21swapper/320:55:353
132523299151,6cyclictest0-21swapper/319:10:343
132523299143,5cyclictest0-21swapper/321:45:263
132523299143,5cyclictest0-21swapper/319:25:373
132523299143,5cyclictest0-21swapper/319:25:373
132523299143,4cyclictest0-21swapper/319:45:373
132523299142,8cyclictest0-21swapper/323:15:363
132523299142,8cyclictest0-21swapper/323:15:363
132523299142,8cyclictest0-21swapper/323:10:283
132523299142,8cyclictest0-21swapper/321:55:323
132523299142,8cyclictest0-21swapper/321:55:323
132523299142,8cyclictest0-21swapper/300:20:353
132523299142,7cyclictest0-21swapper/323:05:293
132523299142,6cyclictest0-21swapper/320:45:013
132523299142,6cyclictest0-21swapper/320:45:013
132523299142,6cyclictest0-21swapper/319:20:323
132523299142,6cyclictest0-21swapper/319:15:313
132523299141,8cyclictest0-21swapper/322:30:353
132523299141,8cyclictest0-21swapper/322:15:383
132523299141,8cyclictest0-21swapper/322:00:343
132523299141,7cyclictest0-21swapper/300:00:263
132523299141,6cyclictest0-21swapper/321:05:333
132523299141,6cyclictest0-21swapper/320:45:373
132523299132,7cyclictest0-21swapper/300:25:253
132523299132,6cyclictest0-21swapper/323:50:293
132523299131,8cyclictest0-21swapper/322:35:363
132523299131,7cyclictest0-21swapper/300:10:233
132523299131,6cyclictest0-21swapper/322:55:243
132523299131,6cyclictest0-21swapper/320:50:313
132523299131,6cyclictest0-21swapper/320:35:373
132523299131,6cyclictest0-21swapper/319:50:373
132523299125,5cyclictest0-21swapper/322:05:403
132523299125,4cyclictest0-21swapper/323:55:383
132523299121,7cyclictest0-21swapper/322:40:273
132523299121,7cyclictest0-21swapper/322:20:243
132523299121,7cyclictest0-21swapper/300:30:303
132523299121,6cyclictest0-21swapper/321:15:253
132523299121,6cyclictest0-21swapper/320:05:363
132523299112,5cyclictest0-21swapper/320:00:013
132523299111,7cyclictest0-21swapper/322:45:053
132523299111,6cyclictest0-21swapper/320:30:303
132523299102,5cyclictest0-21swapper/319:30:343
132523299101,5cyclictest0-21swapper/320:10:343
132523299101,5cyclictest0-21swapper/319:35:323
13252329992,4cyclictest0-21swapper/320:25:253
13252329991,6cyclictest0-21swapper/319:10:013
13252329991,5cyclictest0-21swapper/319:40:273
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional