You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-08 - 17:57

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot3s.osadl.org (updated Wed Jul 08, 2026 12:45:25)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1495548998271,8cyclictest12-21ksoftirqd/010:55:020
1495553997523,12cyclictest275-21dbus-daemon12:40:001
1495553997336,12cyclictest1-21systemd11:25:021
1495553997226,10cyclictest1496760-21sshd07:32:121
149554899719,54cyclictest12-21ksoftirqd/012:17:520
1495557996929,35cyclictest1587678-21wc12:10:012
1495548996947,8cyclictest12-21ksoftirqd/011:39:430
149554899687,55cyclictest1537606-21systemd10:25:000
1495553996723,16cyclictest275-21dbus-daemon07:30:011
1495548996620,4cyclictest12-21ksoftirqd/009:34:010
1495548996620,41cyclictest275-21dbus-daemon10:45:020
1495557996524,37cyclictest1545556-21sh10:41:292
149555799642,18cyclictest1517596-21ssh09:42:592
149555799642,15cyclictest1589937-21munin-run12:15:012
149555799642,15cyclictest1589937-21munin-run12:15:012
149555799641,5cyclictest0-21swapper/210:03:522
149555799641,14cyclictest1506585-21munin-run09:20:002
1495553996428,9cyclictest23-21ksoftirqd/108:09:591
1495548996421,39cyclictest1527467-21sshd10:03:420
1495548996415,45cyclictest12-21ksoftirqd/009:57:510
1495557996323,35cyclictest1511243-21sh09:29:522
149555799631,9cyclictest1538129-21ssh10:25:532
149555799631,8cyclictest1563375-21ssh11:19:202
149554899633,40cyclictest48-21kcompactd009:10:020
149555799621,9cyclictest1593989-21ssh12:23:232
149555799621,9cyclictest1557631-21ssh11:07:012
149555799621,8cyclictest1542591-21ssh10:35:082
149555799621,7cyclictest1594981-21ssh12:25:112
149555799621,7cyclictest1552692-21ssh10:56:332
149555799621,7cyclictest1511611-21ssh09:30:102
149555799621,15cyclictest1496286-21munin-run07:25:002
149554899622,3cyclictest131rcu_preempt10:26:590
149555799611,9cyclictest1584346-21ssh12:03:102
149555799611,8cyclictest1574214-21ssh11:41:492
149555799611,8cyclictest1524985-21ssh09:58:302
149555799611,8cyclictest1521327-21ssh09:50:332
149555799611,7cyclictest1600985-21ssh12:38:062
149555799611,7cyclictest1536030-21ssh10:21:272
149555799611,7cyclictest1534375-21ssh10:18:152
149555799611,7cyclictest1534375-21ssh10:18:142
149555799611,7cyclictest1529086-21ssh10:07:072
149555799611,6cyclictest1568736-21ssh11:30:092
149555799611,5cyclictest1548190-21ssh10:47:022
149555799611,12cyclictest1500703-21munin-run08:50:022
149555799611,12cyclictest1500703-21munin-run08:50:012
149555799611,10cyclictest1582614-21ssh11:59:462
1495553996130,4cyclictest1500145-21cron08:40:011
149555799602,5cyclictest1498120-21munin-run08:00:012
149555799601,8cyclictest1576197-21ssh11:45:492
149555799601,8cyclictest1571222-21ssh11:35:272
149555799601,8cyclictest1559219-21ssh11:10:082
149555799601,8cyclictest1559219-21ssh11:10:082
149555799601,8cyclictest1540624-21ssh10:31:022
149555799601,8cyclictest1515232-21ssh09:38:042
149555799601,7cyclictest1599111-21ssh12:34:192
149555799601,7cyclictest1599111-21ssh12:34:182
149555799601,7cyclictest1550804-21ssh10:52:452
149555799601,7cyclictest1531026-21ssh10:10:562
149555799601,7cyclictest1519863-21ssh09:47:422
149555799601,7cyclictest1502835-21ssh09:12:012
149555799601,7cyclictest1502835-21ssh09:12:002
149555799601,6cyclictest1565184-21ssh11:23:022
149555799601,6cyclictest1507914-21ssh09:22:392
149555799601,5cyclictest1498844-21sshd08:14:132
149555799601,5cyclictest1498621-21munin-run08:10:012
149555799601,10cyclictest1590496-21ssh12:15:502
1495548996025,7cyclictest12-21ksoftirqd/010:07:520
1495548996024,32cyclictest1544369-21ssh10:39:110
149555799591,7cyclictest1578886-21ssh11:51:402
149555799591,7cyclictest1566911-21ssh11:26:302
149555799591,7cyclictest1555198-21ssh11:01:532
1495553995927,10cyclictest23-21ksoftirqd/111:09:571
1495548995936,17cyclictest3467602-21systemd-journal11:45:010
1495548995928,15cyclictest1515127-21ssh09:37:500
1495548995925,15cyclictest1563672-21ssh11:20:010
149555799582,15cyclictest1499667-21munin-run08:30:012
149555799582,15cyclictest1495757-21munin-run07:15:012
1495548995830,12cyclictest1559028-21ssh11:10:030
1495548995829,13cyclictest1560657-21ssh11:13:320
1495548995829,13cyclictest1560657-21ssh11:13:320
1495548995828,12cyclictest1556584-21ssh11:05:010
149554899573,47cyclictest1568444-21modprobe11:29:590
149554899573,23cyclictest1506539-21latency_hist09:19:590
1495548995726,15cyclictest1592993-21ssh12:20:570
1495548995725,15cyclictest1535203-21ssh10:20:010
1495548995725,15cyclictest1535203-21ssh10:20:000
1495559995639,12cyclictest1-21systemd09:55:013
149555799562,11cyclictest1497058-21sendmail07:40:012
149555799561,8cyclictest1500138-21sshd08:38:272
1495557995612,38cyclictest275-21dbus-daemon08:55:012
1495548995633,18cyclictest1518515-21unin-run09:45:020
1495548995626,14cyclictest1578566-21ssh11:50:590
149555799552,11cyclictest364-21snmpd08:02:542
149555799551,14cyclictest1500416-21sshd08:43:122
1495553995536,14cyclictest1-21systemd09:40:001
1495548995528,12cyclictest1596088-21ssh12:27:370
149554899551,38cyclictest1519392-21ssh09:46:310
1495557995446,5cyclictest1495768-21kworker/2:108:20:022
149555799542,6cyclictest1497323-21munin-run07:45:002
1495548995445,5cyclictest16950irq/224-800000010:15:020
1495548995427,12cyclictest1565277-21ssh11:23:130
1495548995426,12cyclictest1548608-21ssh10:48:040
149555799533,4cyclictest0-21swapper/208:33:552
149555799531,5cyclictest364-21snmpd08:55:202
149555799531,13cyclictest364-21snmpd08:22:002
1495548995335,12cyclictest275-21dbus-daemon08:15:010
1495548995330,13cyclictest1498068-21modprobe08:00:010
1495548995325,13cyclictest1542167-21ssh10:34:430
1495548995324,25cyclictest12-21ksoftirqd/009:10:320
1495548995324,25cyclictest12-21ksoftirqd/009:10:310
149554899531,37cyclictest1581623-21ssh11:57:230
1495557995232,15cyclictest275-21dbus-daemon07:35:012
149555799521,6cyclictest1495982-21sshd07:18:122
149555799521,6cyclictest1495982-21sshd07:18:112
149555799521,5cyclictest1501438-21sshd09:00:142
149555799521,5cyclictest1501438-21sshd09:00:132
149555799521,15cyclictest1495632-21grep07:10:012
1495548995243,4cyclictest12-21ksoftirqd/008:50:020
1495548995243,4cyclictest12-21ksoftirqd/008:50:010
1495548995227,10cyclictest1576883-21ssh11:47:240
1495548995217,20cyclictest151rcuc/012:30:030
1495548995217,20cyclictest151rcuc/012:30:020
1495548995214,34cyclictest12-21ksoftirqd/009:51:100
1495559995130,16cyclictest3467602-21systemd-journal11:10:043
1495559995130,16cyclictest3467602-21systemd-journal11:10:033
149555799511,6cyclictest1-21systemd07:50:012
1495553995133,14cyclictest3467602-21systemd-journal11:50:001
1495548995123,21cyclictest275-21dbus-daemon07:40:000
149554899512,11cyclictest1496389-21latency_hist07:25:010
1495553995035,11cyclictest3467602-21systemd-journal08:15:011
1495553995031,13cyclictest275-21dbus-daemon09:30:001
149554899502,40cyclictest1501474-21modprobe09:05:000
149554899502,40cyclictest1501474-21modprobe09:05:000
1495548995021,14cyclictest1510385-21ssh09:27:480
1495548995017,30cyclictest12-21ksoftirqd/009:23:340
149555999493,3cyclictest131rcu_preempt11:26:003
1495559994932,13cyclictest275-21dbus-daemon12:35:023
1495559994932,13cyclictest275-21dbus-daemon12:35:013
149555999492,3cyclictest131rcu_preempt08:30:023
1495559994912,5cyclictest131rcu_preempt09:10:003
149555799492,4cyclictest364-21snmpd07:51:302
149555799491,11cyclictest364-21snmpd07:25:492
1495553994934,11cyclictest1552108-21sshd10:55:131
1495548994929,15cyclictest1-21systemd07:20:020
1495548994929,15cyclictest1-21systemd07:20:010
1495548994929,14cyclictest3467602-21systemd-journal08:19:590
1495548994924,11cyclictest1501727-21diskmemload12:02:070
1495559994836,7cyclictest1556564-21munin-plugin-st11:05:003
1495559994831,13cyclictest1-21systemd10:25:013
149555799481,6cyclictest3467602-21systemd-journal09:09:272
149555999474,3cyclictest131rcu_preempt11:15:503
1495553994735,8cyclictest1582750-21sendmail12:00:001
1495553994725,15cyclictest275-21dbus-daemon11:45:011
149554899473,11cyclictest309-21in:imuxsock08:35:000
1495548994722,9cyclictest12-21ksoftirqd/008:03:270
1495548994720,12cyclictest1569187-21ssh11:31:140
1495548994720,12cyclictest1553454-21ssh10:58:290
149555999466,3cyclictest131rcu_preempt11:53:323
1495559994631,11cyclictest1497769-21sed07:50:023
1495559994624,17cyclictest275-21dbus-daemon08:10:003
1495559994610,20cyclictest1516309-21ssh09:40:023
1495553994634,10cyclictest23-21ksoftirqd/109:55:031
1495548994632,10cyclictest1496777-21cron07:35:000
1495548994629,12cyclictest309-21in:imuxsock07:50:010
1495548994627,15cyclictest1501176-21sshd08:58:310
149554899462,14cyclictest1587707-21latency_hist12:10:020
1495559994535,7cyclictest37-21ksoftirqd/309:14:543
1495559994535,7cyclictest37-21ksoftirqd/309:14:543
149555999452,4cyclictest131rcu_preempt11:09:093
1495553994532,9cyclictest1594696-21ssh12:25:011
1495553994529,12cyclictest3467602-21systemd-journal09:50:001
149555399452,35cyclictest275-21dbus-daemon12:15:011
149555399452,35cyclictest275-21dbus-daemon12:15:001
149554899454,10cyclictest1601820-21ssh12:40:000
1495548994528,13cyclictest1496517-21sshd07:28:150
1495548994518,11cyclictest1588668-21ssh12:12:100
1495548994518,11cyclictest1588668-21ssh12:12:090
149555999444,3cyclictest131rcu_preempt09:43:223
149555999444,28cyclictest361rcuc/311:35:013
149555999443,3cyclictest131rcu_preempt10:05:003
1495559994431,7cyclictest1554140-21sendmail11:00:013
1495559994430,10cyclictest275-21dbus-daemon10:20:013
1495559994430,10cyclictest275-21dbus-daemon10:20:003
1495559994428,13cyclictest1585254-21wc12:05:013
1495553994428,13cyclictest23-21ksoftirqd/109:05:011
1495553994428,13cyclictest23-21ksoftirqd/109:05:001
1495553994428,12cyclictest23-21ksoftirqd/112:35:011
1495553994428,12cyclictest23-21ksoftirqd/112:35:011
1495548994428,12cyclictest12-21ksoftirqd/008:55:000
149555999434,3cyclictest131rcu_preempt09:56:193
149555999433,3cyclictest131rcu_preempt09:33:393
1495559994322,16cyclictest275-21dbus-daemon08:35:003
149555999431,3cyclictest131rcu_preempt12:23:143
1495553994330,10cyclictest275-21dbus-daemon08:00:001
1495553994326,11cyclictest3467718-21lldpd11:05:031
1495553994319,16cyclictest23-21ksoftirqd/110:05:011
149555399431,3cyclictest131rcu_preempt08:50:021
149555399431,3cyclictest131rcu_preempt08:50:011
149554899434,7cyclictest131rcu_preempt07:45:010
1495548994336,5cyclictest19350irq/34-mmc108:40:010
149555999423,3cyclictest131rcu_preempt11:48:093
1495559994233,6cyclictest1498834-21sshd08:12:143
1495559994231,7cyclictest1597119-21sh12:30:023
149555999421,4cyclictest131rcu_preempt11:59:503
1495553994230,9cyclictest23-21ksoftirqd/112:05:001
149555399422,9cyclictest3467602-21systemd-journal10:25:001
1495553994229,7cyclictest309-21in:imuxsock08:20:001
1495553994213,6cyclictest131rcu_preempt11:15:021
1495553994213,6cyclictest131rcu_preempt11:15:021
1495553994212,6cyclictest131rcu_preempt10:53:141
149554899423,7cyclictest1500433-21modprobe08:45:000
1495548994235,4cyclictest16950irq/224-800000008:30:010
1495548994227,12cyclictest1497811-21sshd07:54:130
1495548994216,6cyclictest131rcu_preempt08:25:010
149555999415,4cyclictest131rcu_preempt10:48:223
1495559994129,9cyclictest37-21ksoftirqd/310:50:413
1495559994126,11cyclictest1501039-21wc08:55:013
1495559994120,17cyclictest1-21systemd11:25:023
149555999411,3cyclictest131rcu_preempt12:19:573
1495553994131,6cyclictest1525637-21cron09:59:591
149554899412,6cyclictest1495740-21cron07:15:000
1495559994033,5cyclictest37-21ksoftirqd/310:09:173
1495559994028,10cyclictest37-21ksoftirqd/311:44:103
1495559994024,13cyclictest275-21dbus-daemon09:20:013
1495559994023,13cyclictest37-21ksoftirqd/310:31:393
149555399406,28cyclictest1513828-21ssh09:35:031
1495553994027,10cyclictest0-21swapper/110:35:031
1495553994025,12cyclictest1506558-21tr09:19:591
1495553994025,10cyclictest1592305-21systemctl12:20:001
149555399401,3cyclictest131rcu_preempt11:37:461
1495559993926,9cyclictest37-21ksoftirqd/309:05:013
1495559993926,9cyclictest37-21ksoftirqd/309:05:003
1495559993926,10cyclictest37-21ksoftirqd/307:18:123
1495559993926,10cyclictest37-21ksoftirqd/307:18:113
149555999392,3cyclictest131rcu_preempt10:35:323
1495559993922,13cyclictest1-21systemd10:30:003
149555999391,3cyclictest131rcu_preempt12:35:183
1495553993931,4cyclictest23-21ksoftirqd/107:45:021
1495553993930,4cyclictest1495743-21munin-plugin-st07:15:011
1495553993929,5cyclictest309-21in:imuxsock07:25:001
1495553993928,9cyclictest23-21ksoftirqd/110:35:411
1495553993927,9cyclictest23-21ksoftirqd/110:16:011
1495553993927,9cyclictest23-21ksoftirqd/110:16:011
1495553993927,9cyclictest1547110-21systemctl10:45:011
1495553993927,10cyclictest23-21ksoftirqd/108:05:031
1495553993926,9cyclictest23-21ksoftirqd/111:30:241
1495553993925,11cyclictest23-21ksoftirqd/111:17:401
1495553993924,11cyclictest23-21ksoftirqd/111:50:191
1495553993920,15cyclictest23-21ksoftirqd/110:10:001
149555399391,4cyclictest0-21swapper/110:14:541
1495553993912,6cyclictest131rcu_preempt10:49:361
149555999387,5cyclictest131rcu_preempt10:11:363
149555999383,4cyclictest131rcu_preempt11:37:133
149555999383,11cyclictest37-21ksoftirqd/310:45:003
1495559993828,8cyclictest37-21ksoftirqd/308:55:133
1495559993828,8cyclictest37-21ksoftirqd/308:50:033
1495559993828,8cyclictest37-21ksoftirqd/308:50:023
1495559993827,8cyclictest37-21ksoftirqd/312:10:013
1495559993825,10cyclictest37-21ksoftirqd/309:25:023
1495553993825,9cyclictest3467718-21lldpd10:30:011
1495553993820,15cyclictest23-21ksoftirqd/109:25:001
149555999373,7cyclictest37-21ksoftirqd/312:10:453
149555999373,7cyclictest37-21ksoftirqd/312:10:453
149555999373,3cyclictest131rcu_preempt09:29:203
149555999372,7cyclictest37-21ksoftirqd/308:45:023
1495559993727,8cyclictest37-21ksoftirqd/309:45:093
1495553993727,5cyclictest1501197-21sendmail09:00:001
1495553993726,9cyclictest23-21ksoftirqd/109:45:001
1495553993726,6cyclictest309-21in:imuxsock07:40:001
1495553993724,10cyclictest1504128-21systemctl09:15:011
1495553993724,10cyclictest1504128-21systemctl09:15:001
149555399372,3cyclictest131rcu_preempt12:25:511
149555399371,3cyclictest131rcu_preempt12:10:001
149554899371,2cyclictest0-21swapper/008:10:030
1495559993629,4cyclictest1498051-21sshd07:57:143
1495559993628,5cyclictest309-21in:imuxsock08:40:013
1495559993626,7cyclictest37-21ksoftirqd/307:24:593
1495559993625,9cyclictest37-21ksoftirqd/308:15:133
1495559993624,9cyclictest1495576-21sed07:10:023
149555999362,29cyclictest275-21dbus-daemon07:15:003
1495553993624,10cyclictest1497541-21sshd07:45:111
1495553993622,10cyclictest23-21ksoftirqd/111:25:301
1495548993611,13cyclictest0-21swapper/007:10:020
1495559993523,8cyclictest3467602-21systemd-journal07:37:113
149555399352,5cyclictest131rcu_preempt09:09:591
149555999344,3cyclictest131rcu_preempt08:05:003
1495559993425,6cyclictest37-21ksoftirqd/307:30:013
1495559993420,10cyclictest3467602-21systemd-journal07:45:023
149555999341,4cyclictest131rcu_preempt08:25:003
1495553993424,7cyclictest3467602-21systemd-journal08:21:311
1495553993421,9cyclictest1499618-21sshd08:25:141
1495553993325,6cyclictest23-21ksoftirqd/107:55:011
1495553993322,8cyclictest23-21ksoftirqd/107:20:031
1495553993322,8cyclictest23-21ksoftirqd/107:20:021
1495553993319,9cyclictest1-21systemd08:55:001
149555999327,20cyclictest1496793-21cut07:35:013
1495553993124,4cyclictest3467562-21sshd08:40:131
149555999304,3cyclictest131rcu_preempt07:54:593
149555399305,22cyclictest1499906-21munin-plugin-st08:35:001
149555399284,3cyclictest131rcu_preempt07:10:011
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional