You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-24 - 08:50

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack2slot7s.osadl.org (updated Fri Jul 24, 2026 00:43:11)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1603229995546,6cyclictest1742083-21cups-browsed00:15:481
1603228995540,8cyclictest12650irq/32-eth022:00:360
1603228995540,8cyclictest12650irq/32-eth022:00:360
1603228995525,23cyclictest1628186-21kworker/u8:11+rpciod21:30:450
1603230995044,3cyclictest0-21swapper/223:45:012
1603228994735,7cyclictest0-21swapper/020:30:460
1603229994636,6cyclictest1673440-21cpuspeed22:05:371
1603229994636,6cyclictest1003-21mta-sts-daemon22:01:521
1603229994636,6cyclictest1003-21mta-sts-daemon22:01:521
1603229994536,6cyclictest1746409-21sendmail_mailqu00:05:561
1603230994335,4cyclictest0-21swapper/222:00:012
1603230994335,4cyclictest0-21swapper/222:00:012
1603229994336,4cyclictest1742029-21idleruntime-cro00:00:001
1603229994336,2cyclictest0-21swapper/123:25:581
1603229994335,5cyclictest265-21systemd-journal21:05:011
1603229994333,7cyclictest1749702-21unixbench-2d00:10:581
1603229994331,9cyclictest1694322-21/usr/sbin/munin22:40:381
1603228994334,6cyclictest1675816-21ssh22:10:000
1603228994329,4cyclictest1617228-21fschecks_time19:55:390
1603228994325,13cyclictest12650irq/32-eth021:10:510
1603230994229,10cyclictest1683187-21proc_pri22:20:552
1603229994233,5cyclictest1001-21mta-sts-daemon22:30:061
1603228994218,19cyclictest151ktimers/021:50:500
1603230994133,5cyclictest1766201-21systemd-run00:40:022
1603229994134,4cyclictest1002-21mta-sts-daemon22:46:131
1603229994131,7cyclictest1619737-21sh20:05:011
1603228994136,2cyclictest0-21swapper/019:25:470
1603228994128,3cyclictest0-21swapper/022:15:010
160323099403,32cyclictest1664756-21ssh21:50:492
160323099403,30cyclictest1698046-21ssh22:45:472
1603230994031,6cyclictest421-21dbus-daemon22:30:022
1603229994034,3cyclictest1624180-21kworker/u8:8-xprtiod21:10:011
1603229994033,4cyclictest1725113-21kworker/u8:1-nfsiod00:05:021
1603229994033,4cyclictest1642336-21latency_hist21:15:021
1603229994032,5cyclictest1648487-21latency_hist21:25:021
1603229994031,5cyclictest1607748-21systemd-run19:25:011
1603229994029,8cyclictest420-21avahi-daemon22:35:511
160322999401,30cyclictest665-21snmpd19:35:461
160322999400,7cyclictest28-21ksoftirqd/123:05:531
160322899402,29cyclictest1743144-21sh00:00:450
160323099392,30cyclictest1692035-21diskmemload22:35:462
1603229993932,3cyclictest0-21swapper/121:30:011
1603229993930,6cyclictest265-21systemd-journal23:45:011
160322999392,31cyclictest1701123-21ssh22:50:511
1603228993931,5cyclictest12650irq/32-eth000:15:530
1603228993931,5cyclictest0-21swapper/022:35:510
1603228993929,7cyclictest1734429-21switchtime23:45:550
1603228993928,8cyclictest1684270-21sshd22:22:570
1603228993927,4cyclictest1627130-21unixbench_singl20:25:560
1603228993926,8cyclictest1755561-21switchtime00:20:570
1603230993831,3cyclictest0-21swapper/221:10:502
1603230993829,6cyclictest1-21systemd19:15:012
1603230993829,6cyclictest1-21systemd19:15:012
1603230993828,4cyclictest0-21swapper/200:10:482
1603229993830,5cyclictest1663810-21latency_hist21:50:031
1603229993830,5cyclictest1-21systemd19:15:011
1603229993830,5cyclictest1-21systemd19:15:011
1603229993830,4cyclictest1003-21mta-sts-daemon00:25:421
1603229993826,9cyclictest1667274-21cpu21:55:371
1603229993826,9cyclictest1667274-21cpu21:55:371
1603229993826,9cyclictest1655035-21cpu21:35:361
1603230993730,3cyclictest0-21swapper/200:20:012
1603230993729,3cyclictest0-21swapper/220:55:012
1603230993728,6cyclictest1659265-21unixbench_singl21:40:592
1603229993726,8cyclictest1622818-21latency_hist20:15:021
160322999372,25cyclictest1713061-21rm23:10:471
1603228993730,4cyclictest0-21swapper/019:25:000
1603228993728,6cyclictest1758863-21unixbench_singl00:25:580
1603228993728,6cyclictest1749372-21processes00:10:540
1603228993719,11cyclictest12650irq/32-eth000:35:440
160323099364,27cyclictest1716048-21ssh23:15:452
1603230993630,3cyclictest0-21swapper/221:40:012
1603230993628,5cyclictest1725415-21timerandwakeup23:30:562
1603230993628,5cyclictest1651430-21systemd-run21:30:012
1603230993628,5cyclictest1628713-21grep20:35:012
1603230993627,6cyclictest1703354-21aten_r2power_cu22:55:352
1603229993631,2cyclictest0-21swapper/100:35:011
1603228993630,4cyclictest1686018-21sh22:25:450
1603228993629,3cyclictest0-21swapper/023:35:500
1603228993628,5cyclictest1725201-21proc_pri23:30:520
1603228993628,5cyclictest1668008-21mailstats21:55:540
1603228993628,5cyclictest1668008-21mailstats21:55:540
1603228993627,7cyclictest1742090-40logrotate00:00:020
1603228993627,7cyclictest1648364-21ssh21:25:000
1603228993624,7cyclictest12750irq/33-eth021:15:490
160322899362,26cyclictest1649459-21ssh21:25:490
160323099353,27cyclictest1743253-21ssh00:00:492
1603230993529,3cyclictest0-21swapper/222:15:012
1603229993529,3cyclictest1682998-21modprobe22:20:471
1603229993525,4cyclictest0-21swapper/122:25:371
1603229993524,4cyclictest171rcu_preempt20:20:551
1603229993523,7cyclictest0-21swapper/123:46:301
1603229993518,13cyclictest421-21dbus-daemon23:00:491
1603228993519,4cyclictest0-21swapper/021:40:010
1603230993428,3cyclictest0-21swapper/220:40:002
1603230993428,3cyclictest0-21swapper/220:40:002
1603230993428,3cyclictest0-21swapper/219:55:012
1603230993426,5cyclictest1710497-21timerwakeupswit23:05:572
1603230993426,3cyclictest0-21swapper/223:10:482
1603230993425,6cyclictest1760161-21systemd-run00:30:012
160322999345,18cyclictest1718771-21awk23:20:421
1603229993427,4cyclictest1633744-21gltestperf20:50:391
1603229993427,3cyclictest0-21swapper/121:40:541
1603229993425,6cyclictest485-21in:imuxsock21:20:001
1603229993425,4cyclictest0-21swapper/122:10:471
1603228993427,5cyclictest1-21systemd23:20:010
1603228993425,5cyclictest151ktimers/022:15:570
1603228993422,9cyclictest0-21swapper/023:00:010
160322899342,24cyclictest1605611-21sshd19:15:470
160322899342,24cyclictest1605611-21sshd19:15:470
1603228993421,10cyclictest0-21swapper/023:21:130
1603230993326,3cyclictest0-21swapper/223:35:442
1603230993324,5cyclictest1688900-21grep22:30:422
1603230993324,4cyclictest0-21swapper/222:05:432
1603229993326,3cyclictest0-21swapper/121:55:011
1603229993325,5cyclictest1633116-21unixbench_multi20:45:591
160322999332,24cyclictest1716134-21ssh23:15:481
160322999331,24cyclictest1725112-21ssh23:30:491
160322999330,14cyclictest28-21ksoftirqd/120:30:421
1603228993326,3cyclictest0-21swapper/023:55:010
1603228993321,9cyclictest0-21swapper/023:45:020
1603230993225,4cyclictest1719680-21unixbench_singl23:21:132
1603230993222,7cyclictest1003-21mta-sts-daemon19:45:532
160322999323,23cyclictest1764208-21ssh00:35:441
1603229993225,4cyclictest1636196-21munin-run21:00:011
1603229993225,4cyclictest1000-21mta-sts-daemon19:25:451
1603229993225,3cyclictest0-21swapper/121:30:541
1603229993217,3cyclictest171rcu_preempt22:55:511
1603228993226,3cyclictest0-21swapper/022:45:390
1603228993226,3cyclictest0-21swapper/021:05:020
1603228993224,4cyclictest0-21swapper/022:40:380
160322899322,26cyclictest1639208-21diskmemload00:05:520
160322899322,23cyclictest1710139-21ssh23:05:490
1603230993126,2cyclictest0-21swapper/221:15:452
1603230993124,3cyclictest0-21swapper/222:00:352
1603230993124,3cyclictest0-21swapper/222:00:352
1603230993123,4cyclictest0-21swapper/223:05:012
1603230993122,3cyclictest0-21swapper/223:25:492
1603230993120,7cyclictest351ktimers/219:30:502
1603230993120,6cyclictest1646502-21rm21:20:522
1603229993125,3cyclictest0-21swapper/119:45:011
1603229993123,5cyclictest1626251-21fschecks_time20:25:381
160322999312,23cyclictest1680006-21sh22:15:531
1603229993117,7cyclictest0-21swapper/120:05:491
160322899315,11cyclictest12650irq/32-eth020:25:010
1603228993126,3cyclictest1000-21mta-sts-daemon23:27:020
1603228993125,4cyclictest1634694-21cron20:55:010
1603228993125,3cyclictest1661525-21missed_timers21:45:420
160322899312,25cyclictest1761307-21ssh00:30:480
1603228993118,10cyclictest0-21swapper/019:40:010
1603230993025,2cyclictest0-21swapper/221:35:022
1603230993023,4cyclictest1001-21mta-sts-daemon23:52:292
1603230993023,4cyclictest1000-21mta-sts-daemon23:48:522
1603230993022,5cyclictest1680522-21ssh22:16:052
1603230993021,5cyclictest1000-21mta-sts-daemon20:00:012
1603230993020,8cyclictest1605216-21fschecks_count19:15:392
1603230993020,8cyclictest1605216-21fschecks_count19:15:392
1603229993023,3cyclictest0-21swapper/100:25:011
1603229993022,5cyclictest1002-21mta-sts-daemon23:50:251
1603229993020,3cyclictest171rcu_preempt19:15:481
1603229993020,3cyclictest171rcu_preempt19:15:481
160322999301,4cyclictest171rcu_preempt19:45:511
160322899308,17cyclictest665-21snmpd19:40:470
160322899303,18cyclictest1688955-21diskmemload22:30:440
1603228993021,6cyclictest0-21swapper/022:50:360
160323099293,6cyclictest171rcu_preempt23:55:442
1603230992922,4cyclictest0-21swapper/220:40:472
160323099292,21cyclictest1761217-21ssh00:30:452
1603230992920,6cyclictest0-21swapper/222:40:472
1603230992919,8cyclictest1635506-21needreboot20:55:432
1603230992919,5cyclictest0-21swapper/220:10:462
160322999291,6cyclictest261rcuc/120:15:441
1603228992922,4cyclictest1658048-21aten2.4-expect21:40:370
1603230992822,3cyclictest0-21swapper/221:05:482
160323099282,19cyclictest1661803-21ssh21:45:522
1603230992821,4cyclictest1607718-21cron19:25:002
1603230992821,3cyclictest0-21swapper/222:50:462
1603230992820,5cyclictest0-21swapper/221:05:012
1603230992818,8cyclictest1626236-21fschecks_count20:25:372
160322999283,15cyclictest420-21avahi-daemon19:50:471
160322999281,6cyclictest0-21swapper/120:40:501
1603229992814,4cyclictest171rcu_preempt20:35:461
1603229992814,4cyclictest171rcu_preempt20:35:461
1603229992814,3cyclictest171rcu_preempt19:30:491
1603228992822,3cyclictest0-21swapper/019:10:370
1603228992822,3cyclictest0-21swapper/019:10:370
1603228992821,3cyclictest0-21swapper/023:10:480
1603228992821,3cyclictest0-21swapper/019:50:470
1603228992819,6cyclictest0-21swapper/019:45:480
160322899281,19cyclictest665-21snmpd20:00:510
1603230992719,3cyclictest0-21swapper/200:20:132
1603230992718,3cyclictest0-21swapper/200:05:512
160322999278,3cyclictest171rcu_preempt19:55:571
160322999274,13cyclictest1728203-21ssh23:35:511
1603228992722,3cyclictest487-21rs:main0
1603228992717,8cyclictest1623511-21needreboot20:15:420
1603228992717,7cyclictest0-21swapper/020:40:010
1603228992717,7cyclictest0-21swapper/020:40:010
1603228992717,6cyclictest12650irq/32-eth020:10:460
1603228992713,11cyclictest0-21swapper/023:00:400
160323099262,18cyclictest420-21avahi-daemon20:05:492
1603229992618,6cyclictest421-21dbus-daemon19:10:011
1603229992618,6cyclictest421-21dbus-daemon19:10:011
160322899260,23cyclictest1631703-21kworker/u8:5+xprtiod20:45:590
1603230992519,4cyclictest1586071-21kworker/u8:14+rpciod19:10:012
1603230992519,4cyclictest1586071-21kworker/u8:14+rpciod19:10:012
1603228992510,12cyclictest4066570-21cups-browsed20:05:490
160323099249,11cyclictest351ktimers/219:35:552
1603230992418,3cyclictest0-21swapper/220:20:012
1603230992413,8cyclictest351ktimers/220:20:472
1603230992412,9cyclictest1632174-21cpuspeed20:45:392
1603228992411,4cyclictest1000-21mta-sts-daemon19:30:410
1603228992316,4cyclictest0-21swapper/021:05:510
1603230992214,6cyclictest1612470-21aten_r2power_cu19:40:352
1603230992214,6cyclictest1608342-21mii-tool19:25:412
1603228992216,4cyclictest623-21sshd20:57:510
1603230992116,2cyclictest0-21swapper/220:05:012
160323199191,9cyclictest0-21swapper/322:00:363
160323199191,9cyclictest0-21swapper/322:00:363
160323199191,7cyclictest0-21swapper/320:40:443
160323199183,10cyclictest0-21swapper/321:05:483
160323199182,7cyclictest0-21swapper/321:20:523
160323199181,9cyclictest0-21swapper/300:05:483
160323199181,8cyclictest0-21swapper/323:40:543
160323199181,7cyclictest0-21swapper/323:50:493
160323199173,9cyclictest0-21swapper/321:30:443
160323199172,8cyclictest0-21swapper/323:05:483
160323199172,6cyclictest0-21swapper/321:15:493
160322899170,14cyclictest1603417-21latency_hist19:10:020
160322899170,14cyclictest1603417-21latency_hist19:10:020
160323199162,7cyclictest0-21swapper/323:00:493
160323199161,8cyclictest0-21swapper/322:50:473
160323199161,8cyclictest0-21swapper/321:10:453
160323199161,8cyclictest0-21swapper/300:20:483
160323199153,4cyclictest0-21swapper/319:10:473
160323199153,4cyclictest0-21swapper/319:10:473
160323199152,8cyclictest0-21swapper/300:15:483
160323199152,6cyclictest0-21swapper/323:20:363
160323199151,9cyclictest0-21swapper/323:46:303
160323199151,8cyclictest0-21swapper/323:15:393
160323199151,8cyclictest0-21swapper/322:25:473
160323199151,8cyclictest0-21swapper/321:35:403
160323199151,7cyclictest0-21swapper/300:25:463
160323199151,6cyclictest0-21swapper/320:05:473
160323199151,10cyclictest0-21swapper/322:10:533
160323199145,6cyclictest0-21swapper/321:55:483
160323199145,6cyclictest0-21swapper/321:55:483
160323199143,6cyclictest0-21swapper/322:45:443
160323199143,6cyclictest0-21swapper/300:35:583
160323199142,8cyclictest0-21swapper/300:31:463
160323199142,7cyclictest0-21swapper/323:10:483
160323199142,7cyclictest0-21swapper/322:30:493
160323199142,7cyclictest0-21swapper/322:05:433
160323199142,7cyclictest0-21swapper/321:50:473
160323199142,7cyclictest0-21swapper/321:40:473
160323199142,7cyclictest0-21swapper/319:40:023
160323199142,7cyclictest0-21swapper/300:00:013
160323199141,8cyclictest0-21swapper/322:40:513
160323199141,8cyclictest0-21swapper/321:25:483
160323199141,7cyclictest0-21swapper/323:30:483
160323199141,7cyclictest0-21swapper/319:25:513
160323199141,6cyclictest0-21swapper/320:35:493
160323199141,6cyclictest0-21swapper/320:35:493
160323199133,6cyclictest0-21swapper/322:55:533
160323199133,4cyclictest0-21swapper/319:30:483
160323199132,7cyclictest0-21swapper/323:35:403
160323199132,7cyclictest0-21swapper/322:20:413
160323199132,7cyclictest0-21swapper/321:45:473
160323199132,5cyclictest0-21swapper/319:40:483
160323199131,8cyclictest0-21swapper/319:45:493
160323199131,7cyclictest0-21swapper/323:25:473
160323199131,7cyclictest0-21swapper/322:35:453
160323199131,7cyclictest0-21swapper/300:00:503
160323199131,6cyclictest0-21swapper/321:00:513
160323199131,6cyclictest0-21swapper/320:30:503
160323199131,6cyclictest0-21swapper/319:15:463
160323199131,6cyclictest0-21swapper/319:15:463
160323199131,6cyclictest0-21swapper/300:10:373
160323199131,5cyclictest0-21swapper/322:15:053
160323199131,5cyclictest0-21swapper/320:55:463
160323199121,6cyclictest0-21swapper/320:45:483
160323199121,6cyclictest0-21swapper/320:15:483
160323199121,6cyclictest0-21swapper/320:00:463
160323199121,6cyclictest0-21swapper/319:55:473
160323199121,5cyclictest0-21swapper/320:25:443
160323199121,5cyclictest0-21swapper/320:20:423
160323199121,5cyclictest0-21swapper/320:10:433
160323199111,6cyclictest0-21swapper/319:50:473
160323199111,6cyclictest0-21swapper/319:20:483
160323199101,6cyclictest0-21swapper/320:50:473
16032319981,4cyclictest0-21swapper/319:10:023
16032319981,4cyclictest0-21swapper/319:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional