You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-15 - 03:03

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack2slot7s.osadl.org (updated Wed Jul 15, 2026 00:43:16)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1420195995343,6cyclictest1503122-21proc_pri22:25:481
1420196995140,5cyclictest0-21swapper/222:45:442
1420196994938,7cyclictest1551101-21fschecks_count23:45:372
1420196994840,5cyclictest0-21swapper/223:10:372
1420196994837,6cyclictest1468432-21unin-run21:30:012
1420195994841,4cyclictest0-21swapper/100:00:011
1420195994635,8cyclictest1500554-21/usr/sbin/munin22:20:541
1420194994636,7cyclictest1550438-21rm23:45:010
142019499462,36cyclictest151ktimers/022:10:470
1420195994535,6cyclictest1-21systemd23:20:011
1420194994536,6cyclictest3850238-21cups-browsed22:55:410
1420194994536,6cyclictest3850238-21cups-browsed22:55:410
1420196994434,6cyclictest1518514-21timerandwakeup22:50:532
1420196994434,6cyclictest1518514-21timerandwakeup22:50:532
142019499441,36cyclictest3850238-21cups-browsed19:45:460
142019699435,31cyclictest1469701-21awk21:30:492
1420196994336,4cyclictest1514004-21debian-sa122:45:012
1420196994335,5cyclictest1480606-21munin-plugin-st21:50:012
1420196994334,5cyclictest1571464-21sendmail00:20:012
1420195994335,5cyclictest1536011-21if_eth023:20:361
1420194994317,18cyclictest151ktimers/019:25:410
1420194994317,18cyclictest151ktimers/019:25:410
1420194994233,6cyclictest1-21systemd22:30:020
1420194994217,20cyclictest1475615-21ssh21:40:410
1420196994134,4cyclictest1559503-40logrotate00:00:022
142019699412,24cyclictest1475795-21grep21:40:482
1420194994133,5cyclictest1457813-21grep21:10:540
1420196994032,5cyclictest1504916-21systemd-run22:30:012
1420196994029,8cyclictest1583456-21/usr/sbin/munin00:40:022
142019699402,6cyclictest171rcu_preempt23:35:472
142019699398,21cyclictest0-21swapper/223:40:432
1420196993926,9cyclictest1572932-21timerandwakeup00:20:532
1420195993930,6cyclictest420-21avahi-daemon00:23:511
1420195993929,8cyclictest1430687-21/usr/sbin/munin19:45:011
1420195993929,6cyclictest1559552-21cups-browsed00:28:471
1420194993933,4cyclictest1565462-21cron00:10:010
1420194993933,4cyclictest1565462-21cron00:10:010
1420196993830,5cyclictest1529185-21latency_hist23:10:012
1420196993829,5cyclictest1424684-21cron19:25:012
1420196993827,7cyclictest1509516-21unixbench-2d22:35:512
1420196993826,9cyclictest1497195-21switchtime22:15:532
1420196993823,9cyclictest1454124-21ntpq21:00:452
142019699382,26cyclictest1481746-21ssh21:50:442
1420196993812,8cyclictest1526029-21sh23:04:452
1420196993811,18cyclictest0-21swapper/219:15:432
1420195993830,5cyclictest1480637-21ssh21:50:021
1420195993830,5cyclictest1456203-21systemd-run21:10:011
1420195993829,6cyclictest3850238-21cups-browsed23:54:341
1420194993831,5cyclictest1447093-21head20:35:530
1420194993830,5cyclictest420-21avahi-daemon22:54:570
1420194993830,5cyclictest420-21avahi-daemon22:54:570
1420194993829,6cyclictest420-21avahi-daemon00:37:250
1420194993825,10cyclictest1421693-21systemd-run19:15:010
1420194993819,12cyclictest12650irq/32-eth021:15:440
142019699373,25cyclictest1484814-21sh21:55:462
142019699373,25cyclictest1484814-21sh21:55:462
1420196993729,5cyclictest1-21systemd21:20:012
1420196993723,8cyclictest1440568-21ntpq20:15:432
1420195993730,4cyclictest1421661-21kworker/u8:14-xprtiod21:00:011
1420195993728,6cyclictest420-21avahi-daemon22:08:561
1420195993728,5cyclictest420-21avahi-daemon21:44:201
1420194993731,3cyclictest1001-21mta-sts-daemon00:23:520
1420194993730,4cyclictest1527385-21proc_pri23:05:490
1420194993720,13cyclictest12650irq/32-eth023:55:400
1420194993715,17cyclictest12750irq/33-eth023:20:420
142019699364,26cyclictest1463389-21ssh21:20:412
1420196993627,6cyclictest1436678-21cron20:05:012
1420196993627,4cyclictest0-21swapper/223:30:412
1420195993630,3cyclictest0-21swapper/121:40:011
1420195993629,4cyclictest1571434-21ssh00:20:011
1420195993629,3cyclictest0-21swapper/123:10:441
1420195993628,5cyclictest1442710-21systemd-run20:25:021
1420195993627,6cyclictest1559552-21cups-browsed00:31:441
1420195993626,6cyclictest420-21avahi-daemon22:47:261
1420194993628,5cyclictest750-21ntpd22:05:520
1420194993622,11cyclictest1510960-21grep22:40:010
1420194993611,20cyclictest627-21lldpd22:24:190
1420196993527,4cyclictest0-21swapper/200:10:382
1420196993526,7cyclictest694-21cron22:05:002
1420196993522,8cyclictest1428580-21ntpq19:35:422
1420196993520,10cyclictest1421039-21ntpq19:10:442
1420195993528,4cyclictest1553436-21idleruntime-cro23:50:011
1420195993527,5cyclictest1483644-21munin-plugin-st21:55:011
1420195993526,6cyclictest420-21avahi-daemon22:15:431
1420195993522,10cyclictest1545197-21latency23:35:371
1420194993529,4cyclictest1498830-21idleruntime-cro22:20:010
1420194993529,4cyclictest1435183-21tr20:00:010
1420194993529,3cyclictest0-21swapper/021:30:450
1420194993528,3cyclictest12650irq/32-eth023:30:480
1420194993527,5cyclictest1484878-21sh21:55:490
1420194993527,5cyclictest1484878-21sh21:55:490
1420194993527,5cyclictest0-21swapper/019:30:390
1420194993527,5cyclictest0-21swapper/019:30:390
142019699343,25cyclictest1563539-21ssh00:05:422
142019699343,25cyclictest1563539-21ssh00:05:422
1420196993426,5cyclictest1425204-21fschecks_count19:25:342
1420196993426,5cyclictest1425204-21fschecks_count19:25:342
1420196993426,4cyclictest1474524-21munin-run21:40:012
1420196993420,8cyclictest1446516-21ntpq20:35:402
1420196993415,13cyclictest351ktimers/223:25:462
1420196993410,8cyclictest1001-21mta-sts-daemon00:32:092
1420195993428,3cyclictest0-21swapper/123:10:011
1420195993425,3cyclictest0-21swapper/123:25:451
142019499343,25cyclictest1478639-21ssh21:45:400
1420194993428,4cyclictest1578091-21cpuspeed00:30:310
1420194993427,4cyclictest1562475-21cron00:05:010
1420194993427,4cyclictest1526125-21munin-plugin-st23:05:000
1420194993426,5cyclictest420-21avahi-daemon23:17:170
1420194993417,10cyclictest1472732-21ssh21:35:490
1420196993325,5cyclictest0-21swapper/223:20:012
1420196993325,4cyclictest0-21swapper/223:50:322
1420196993323,6cyclictest421-21dbus-daemon22:14:042
142019699332,24cyclictest1521249-21ssh22:55:502
142019699332,24cyclictest1521249-21ssh22:55:502
1420196993321,9cyclictest1445380-21/usr/sbin/munin20:30:522
1420196993320,8cyclictest1434539-21ntpq19:55:412
1420196993319,8cyclictest1452555-21ntpq20:55:452
1420195993326,4cyclictest1454796-21latency_hist21:05:021
1420195993326,4cyclictest0-21swapper/100:05:511
1420195993326,4cyclictest0-21swapper/100:05:511
1420195993325,5cyclictest1548419-21ntp_kernel_err23:40:381
1420195993325,5cyclictest1423795-21/usr/sbin/munin19:20:361
142019599331,26cyclictest1518295-21ssh22:50:491
142019599331,26cyclictest1518295-21ssh22:50:491
1420194993327,4cyclictest1454708-21systemd-run21:05:010
1420194993325,5cyclictest265-21systemd-journal20:25:010
1420194993324,7cyclictest1002-21mta-sts-daemon20:35:020
1420194993323,7cyclictest0-21swapper/021:50:490
1420196993221,7cyclictest1449408-21/usr/sbin/munin20:45:362
1420196993220,8cyclictest1447414-21/usr/sbin/munin20:40:572
1420196993219,9cyclictest1499767-21missed_timers22:20:382
1420195993227,2cyclictest0-21swapper/121:10:541
1420195993226,3cyclictest1000-21mta-sts-daemon22:59:151
1420195993226,3cyclictest1000-21mta-sts-daemon22:59:151
1420195993226,3cyclictest0-21swapper/121:55:321
1420195993226,3cyclictest0-21swapper/121:55:321
1420195993225,4cyclictest1429181-21basename19:40:001
1420195993225,4cyclictest0-21swapper/123:30:361
1420195993224,5cyclictest1431208-21fschecks_time19:45:361
1420195993223,6cyclictest1514042-21latency_hist22:45:021
1420195993221,7cyclictest1559552-21cups-browsed00:13:011
1420194993226,4cyclictest1547365-21sendmail23:40:010
1420194993226,3cyclictest0-21swapper/022:44:500
1420194993223,6cyclictest3850238-21cups-browsed21:28:000
142019499322,24cyclictest1456178-21diskmemload22:30:400
1420194993217,9cyclictest151ktimers/019:50:410
142019699319,7cyclictest1576093-21unixbench_multi00:25:532
142019699315,9cyclictest1424666-21kworker/u8:2+xprtiod20:10:482
1420196993122,4cyclictest1430129-21ntpq19:40:432
1420196993119,8cyclictest1427023-21ntpq19:30:392
1420196993119,8cyclictest1427023-21ntpq19:30:392
1420196993119,7cyclictest1437555-21ntpq20:05:402
1420196993118,9cyclictest1451031-21ntpq20:50:412
1420196993118,9cyclictest1443696-21/usr/sbin/munin20:25:482
1420195993126,2cyclictest0-21swapper/119:15:011
1420195993125,3cyclictest0-21swapper/119:10:011
1420195993124,4cyclictest1435169-21munin-run20:00:011
1420195993124,4cyclictest0-21swapper/119:35:001
1420195993124,4cyclictest0-21swapper/119:35:001
1420195993123,5cyclictest1488148-21xargs22:00:521
1420195993122,6cyclictest1438251-21latency_hist20:10:021
142019599312,21cyclictest1524198-21ssh23:00:431
142019499318,9cyclictest421-21dbus-daemon23:50:430
1420194993125,4cyclictest1465388-21systemd21:25:020
1420194993125,4cyclictest0-21swapper/019:40:000
1420194993124,4cyclictest420-21avahi-daemon20:27:480
142019699309,7cyclictest1456178-21diskmemload22:05:312
1420196993020,7cyclictest0-21swapper/223:20:322
1420196993018,7cyclictest1442169-21ntpq20:20:472
1420196993017,8cyclictest1505959-21ntpq22:30:402
1420196993013,13cyclictest1433041-21/usr/sbin/munin19:50:412
1420196993012,14cyclictest1560744-21ntpq00:01:022
1420195993023,4cyclictest1493154-21aten_r2power_cu22:10:321
1420195993023,3cyclictest0-21swapper/120:25:471
1420195993022,5cyclictest1447172-21munin-run20:40:011
1420194993023,4cyclictest1532180-21munin-plugin-st23:15:010
1420194993012,13cyclictest12650irq/32-eth000:25:410
142019699292,20cyclictest1457286-21ssh21:10:412
1420196992921,3cyclictest1431570-21ntpq19:45:432
1420196992919,5cyclictest1455553-21ntpq21:05:402
142019599297,3cyclictest171rcu_preempt21:20:411
1420195992922,4cyclictest1576618-21kworker/u8:3-rpciod00:37:361
1420194992920,6cyclictest3850238-21cups-browsed23:45:350
1420194992913,11cyclictest421-21dbus-daemon23:25:130
142019599288,3cyclictest171rcu_preempt22:30:351
1420195992822,3cyclictest0-21swapper/122:35:331
1420195992822,3cyclictest0-21swapper/100:00:551
1420195992818,7cyclictest1466810-21switchtime21:25:551
1420195992818,7cyclictest1438518-21aten_r2power_po20:10:321
142019499283,16cyclictest1487736-21rm22:00:390
142019499281,23cyclictest1566660-21ntp_states00:10:500
1420195992723,2cyclictest0-21swapper/120:20:011
1420195992722,2cyclictest0-21swapper/121:30:431
142019599272,18cyclictest1460362-21ssh21:15:441
1420195992720,4cyclictest1436107-21ntp_offset20:00:431
1420195992719,5cyclictest1451520-21grep20:50:531
1420195992719,5cyclictest1423176-21cron19:20:011
1420194992720,4cyclictest0-21swapper/022:48:090
1420194992719,6cyclictest1429833-21kernelversion19:40:340
1420196992621,3cyclictest0-21swapper/219:10:012
1420195992617,7cyclictest1449199-21fschecks_count20:45:331
142019499262,19cyclictest1571444-21munin-run00:20:010
1420194992621,2cyclictest0-21swapper/021:10:010
1420194992613,10cyclictest0-21swapper/020:20:010
1420195992519,3cyclictest0-21swapper/120:35:011
1420195992516,7cyclictest1448563-21unixbench_multi20:40:551
142019499252,19cyclictest3850234-21cupsd20:10:410
1420194992515,8cyclictest1451068-21ntp_kernel_pll_20:50:430
142019499244,13cyclictest428-21mta-sts-daemon20:55:460
142019499241,19cyclictest1424145-21ntp_states19:20:440
1420195992317,3cyclictest0-21swapper/119:30:021
1420195992317,3cyclictest0-21swapper/119:30:021
142019499239,8cyclictest2-21kthreadd20:49:300
142019499239,4cyclictest1435713-21fschecks_time20:00:330
1420194992316,4cyclictest3850238-21cups-browsed19:16:340
142019599222,14cyclictest665-21snmpd19:50:401
1420194992216,3cyclictest151ktimers/020:40:470
1420194992116,2cyclictest0-21swapper/020:05:120
142019799193,9cyclictest0-21swapper/300:25:533
142019799183,10cyclictest0-21swapper/323:50:433
142019799182,10cyclictest0-21swapper/323:20:493
1420194991813,2cyclictest0-21swapper/019:10:010
142019799173,8cyclictest0-21swapper/323:25:413
142019799173,10cyclictest0-21swapper/321:50:363
142019799172,9cyclictest0-21swapper/323:30:443
142019799172,9cyclictest0-21swapper/321:30:423
142019799172,9cyclictest0-21swapper/321:15:313
142019799172,9cyclictest0-21swapper/300:05:493
142019799172,9cyclictest0-21swapper/300:05:493
142019799172,10cyclictest0-21swapper/323:05:493
142019799171,9cyclictest0-21swapper/322:40:463
142019799163,7cyclictest0-21swapper/322:10:363
142019799163,6cyclictest0-21swapper/322:50:413
142019799163,6cyclictest0-21swapper/322:50:413
142019799162,9cyclictest0-21swapper/323:45:503
142019799162,9cyclictest0-21swapper/323:35:463
142019799162,9cyclictest0-21swapper/323:10:353
142019799162,9cyclictest0-21swapper/322:15:493
142019799162,9cyclictest0-21swapper/321:35:473
142019799162,9cyclictest0-21swapper/300:15:463
142019799162,9cyclictest0-21swapper/300:01:003
142019799162,8cyclictest0-21swapper/323:55:403
142019799162,8cyclictest0-21swapper/320:00:463
142019799161,9cyclictest0-21swapper/321:40:483
142019799161,8cyclictest0-21swapper/323:00:403
142019799161,8cyclictest0-21swapper/322:25:413
142019799161,8cyclictest0-21swapper/322:05:523
142019799161,7cyclictest0-21swapper/322:35:333
142019799161,10cyclictest0-21swapper/321:20:313
142019799154,8cyclictest0-21swapper/321:45:403
142019799154,8cyclictest0-21swapper/300:10:483
142019799153,9cyclictest0-21swapper/300:20:353
142019799153,8cyclictest0-21swapper/322:00:393
142019799153,8cyclictest0-21swapper/319:30:443
142019799153,8cyclictest0-21swapper/319:30:443
142019799153,6cyclictest0-21swapper/322:55:443
142019799153,6cyclictest0-21swapper/322:55:443
142019799152,9cyclictest0-21swapper/323:40:383
142019799152,8cyclictest0-21swapper/321:10:393
142019799152,7cyclictest0-21swapper/320:50:423
142019799152,7cyclictest0-21swapper/320:45:423
142019799151,9cyclictest0-21swapper/300:30:393
142019799151,7cyclictest0-21swapper/319:50:443
142019799144,8cyclictest0-21swapper/322:20:323
142019799142,8cyclictest0-21swapper/321:25:423
142019799142,6cyclictest0-21swapper/321:00:443
142019799141,9cyclictest0-21swapper/322:45:383
142019799141,9cyclictest0-21swapper/320:25:473
142019799141,8cyclictest0-21swapper/322:30:363
142019799141,8cyclictest0-21swapper/321:55:473
142019799141,8cyclictest0-21swapper/321:55:473
142019799141,8cyclictest0-21swapper/321:10:003
142019799134,7cyclictest0-21swapper/320:10:413
142019799133,7cyclictest0-21swapper/323:15:363
142019799132,7cyclictest0-21swapper/319:20:473
142019799132,7cyclictest0-21swapper/300:35:343
142019799131,7cyclictest0-21swapper/320:55:433
142019799131,7cyclictest0-21swapper/319:45:463
142019799131,6cyclictest0-21swapper/320:20:443
142019799121,8cyclictest0-21swapper/319:15:483
142019799121,6cyclictest0-21swapper/320:05:403
142019799121,6cyclictest0-21swapper/319:40:383
142019799121,6cyclictest0-21swapper/319:25:423
142019799121,6cyclictest0-21swapper/319:25:423
142019799121,6cyclictest0-21swapper/319:10:443
142019799112,6cyclictest0-21swapper/320:35:493
142019799111,6cyclictest0-21swapper/320:40:413
142019799111,6cyclictest0-21swapper/319:35:413
142019799104,5cyclictest0-21swapper/320:15:413
142019799103,5cyclictest0-21swapper/319:55:363
14201979993,4cyclictest0-21swapper/320:30:403
14201979982,4cyclictest0-21swapper/319:10:013
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional