You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-09 - 06:01

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack2slot7s.osadl.org (updated Sat May 09, 2026 00:43:08)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
323667996454,6cyclictest0-21swapper/123:45:561
323667995341,5cyclictest171rcu_preempt21:15:511
32366799482,35cyclictest477628-21ssh23:35:541
323667994735,6cyclictest0-21swapper/122:05:551
323667994735,6cyclictest0-21swapper/122:05:551
32366799473,36cyclictest501711-21ssh00:05:531
323666994737,7cyclictest421-21dbus-daemon22:05:550
323666994737,7cyclictest421-21dbus-daemon22:05:550
323668994639,3cyclictest0-21swapper/200:20:492
323668994638,3cyclictest456188-21kworker/u8:16-xprtiod23:10:512
32366699464,34cyclictest421371-21ssh22:25:540
323666994632,9cyclictest12350irq/32-eth023:05:590
323668994537,4cyclictest1054-21mta-sts-daemon23:05:542
323666994533,4cyclictest0-21swapper/022:00:010
323668994436,5cyclictest356647-21systemctl21:00:002
323667994434,7cyclictest421-21dbus-daemon23:50:411
32366699448,25cyclictest1056-21mta-sts-daemon00:00:510
323666994435,4cyclictest0-21swapper/000:20:540
323667994323,3cyclictest0-21swapper/123:25:541
323666994336,4cyclictest328902-21missed_timers19:25:450
323666994335,5cyclictest469810-21switchtime23:25:590
323668994233,5cyclictest484292-21systemd-run23:45:012
323668994233,2cyclictest0-21swapper/221:25:502
323668994232,7cyclictest504446-21systemd-run00:10:012
323667994234,4cyclictest0-21swapper/122:40:511
323667994233,6cyclictest409033-21ssh22:10:451
323667994231,6cyclictest0-21swapper/122:45:541
323667994231,6cyclictest0-21swapper/122:45:541
323667994231,4cyclictest0-21swapper/122:25:541
323667994230,8cyclictest2750506-21cups-browsed23:00:561
323667994225,10cyclictest359653-21diskmemload21:50:541
323666994234,4cyclictest432021-21awk22:40:010
323666994231,8cyclictest355159-21systemd-run20:55:010
323668994134,4cyclictest391800-21kworker/u8:7+rpciod21:50:012
323667994132,6cyclictest0-21swapper/123:15:011
32366699412,31cyclictest481633-21ssh23:40:530
323666994118,18cyclictest473507-21ssh23:30:510
323668994031,6cyclictest371794-21latency_hist21:25:022
323667994032,5cyclictest399861-21tr22:00:011
323667994031,6cyclictest1055-21mta-sts-daemon22:24:451
323667994027,10cyclictest445332-21ntp_kernel_err22:55:481
323666994031,6cyclictest151ktimers/020:45:470
323666994028,8cyclictest420-21avahi-daemon00:35:520
32366699401,19cyclictest433331-21ssh22:40:490
32366899394,31cyclictest385064-21ssh21:40:502
32366799396,27cyclictest509868-21ssh00:15:561
323667993933,3cyclictest0-21swapper/122:20:011
323667993931,5cyclictest354166-21fschecks_count20:50:431
32366799392,29cyclictest497785-21ssh00:00:531
323666993923,7cyclictest12350irq/32-eth021:50:520
323666993921,12cyclictest12350irq/32-eth023:35:470
32366899386,24cyclictest445530-21sh22:55:542
323668993832,4cyclictest496326-40logrotate00:00:002
323668993832,4cyclictest496326-40logrotate00:00:002
323668993830,5cyclictest379733-21ssh21:35:012
323668993830,5cyclictest379733-21ssh21:35:012
323668993829,6cyclictest466037-21grep23:20:592
323667993829,6cyclictest420-21avahi-daemon21:10:561
323666993819,9cyclictest12350irq/32-eth022:45:540
323666993819,9cyclictest12350irq/32-eth022:45:540
323666993816,16cyclictest12350irq/32-eth021:25:520
323668993729,5cyclictest485155-21gltestperf23:45:432
323668993729,5cyclictest449861-21xargs23:01:022
323668993727,6cyclictest524570-21systemd-run00:35:012
323668993726,7cyclictest421-21dbus-daemon22:29:452
323668993726,5cyclictest0-21swapper/222:10:472
32366899372,32cyclictest469425-21ssh23:25:472
323667993731,3cyclictest0-21swapper/122:05:011
323667993731,3cyclictest0-21swapper/122:05:011
323667993726,7cyclictest420-21avahi-daemon21:43:331
32366699373,27cyclictest461513-21ssh23:15:510
323666993719,13cyclictest12350irq/32-eth000:15:520
32366699371,30cyclictest409302-21ssh22:10:530
32366899365,27cyclictest425293-21ssh22:30:512
323668993630,3cyclictest0-21swapper/219:20:012
323668993629,4cyclictest512630-21latency_hist00:20:022
323668993629,4cyclictest1054-21mta-sts-daemon22:35:462
323668993628,4cyclictest0-21swapper/222:15:512
323668993623,10cyclictest477276-21latency23:35:432
32366899361,27cyclictest517882-21ssh00:25:532
32366799363,25cyclictest441425-21ssh22:50:511
323667993629,4cyclictest493243-21if_eth023:55:421
323667993629,4cyclictest493243-21if_eth023:55:421
323667993629,3cyclictest0-21swapper/123:15:381
323667993623,10cyclictest659-21snmpd21:00:521
323667993620,12cyclictest421-21dbus-daemon23:05:541
323668993530,2cyclictest0-21swapper/223:20:012
323668993528,4cyclictest331680-21fschecks_time19:35:412
323668993527,5cyclictest405831-21head22:06:012
323668993527,5cyclictest405831-21head22:06:012
323668993526,6cyclictest352178-21true20:45:012
323668993525,7cyclictest439-21systemd-logind22:45:022
323668993525,7cyclictest397450-21switchtime21:55:592
323667993527,5cyclictest329681-21systemd-run19:30:011
323667993523,9cyclictest353057-21ntp_kernel_pll_20:45:501
32366699359,18cyclictest428-21mta-sts-daemon23:10:520
323666993529,3cyclictest0-21swapper/000:30:500
323666993528,4cyclictest341660-21munin-plugin-st20:10:010
323666993524,8cyclictest380835-21missed_timers21:35:440
323666993524,8cyclictest380835-21missed_timers21:35:440
323666993522,6cyclictest12350irq/32-eth023:45:500
323668993427,4cyclictest440910-21ssh22:50:402
323668993426,5cyclictest0-21swapper/220:35:012
323668993426,4cyclictest474147-21unixbench_singl23:31:042
323668993424,6cyclictest2750506-21cups-browsed21:53:382
323668993422,7cyclictest359653-21diskmemload22:00:562
323668993422,7cyclictest359653-21diskmemload22:00:562
323667993426,5cyclictest389707-21sed21:46:031
323667993425,6cyclictest496353-21cups-browsed00:13:071
323667993421,10cyclictest1-21systemd21:30:391
323667993421,10cyclictest1-21systemd21:30:391
323667993416,3cyclictest171rcu_preempt19:30:501
32366699343,25cyclictest493707-21ssh23:55:530
32366699343,25cyclictest493707-21ssh23:55:530
323666993428,4cyclictest371779-21systemd-run21:25:010
323666993427,4cyclictest413085-21netstat22:15:460
323666993418,10cyclictest12350irq/32-eth022:30:520
32366899333,26cyclictest359653-21diskmemload21:15:572
323668993323,7cyclictest340179-21sh20:05:012
32366899332,23cyclictest437486-21ssh22:45:542
32366899332,23cyclictest437486-21ssh22:45:542
323668993318,12cyclictest1051-21mta-sts-daemon23:50:492
32366799335,15cyclictest369142-21ssh21:20:521
32366799333,22cyclictest380963-21ssh21:35:471
32366799333,22cyclictest380963-21ssh21:35:471
323667993326,3cyclictest0-21swapper/122:35:501
323667993325,4cyclictest0-21swapper/100:25:421
32366799331,25cyclictest521937-21rm00:30:551
323666993323,7cyclictest506125-21timerandwakeup00:11:020
323666993322,9cyclictest0-21swapper/023:51:020
323666993318,12cyclictest12350irq/32-eth019:50:550
323666993311,17cyclictest465410-21ssh23:20:470
32366899323,23cyclictest417314-21sh22:20:532
32366899323,22cyclictest2750506-21cups-browsed19:50:552
323668993226,3cyclictest0-21swapper/219:45:012
323668993224,5cyclictest344705-21latency_hist20:20:012
323668993224,4cyclictest0-21swapper/200:40:012
323668993223,6cyclictest359653-21diskmemload00:10:502
32366799323,21cyclictest359653-21diskmemload21:25:541
323667993225,3cyclictest0-21swapper/122:35:011
323667993223,6cyclictest344655-21munin-plugin-st20:20:001
323667993223,6cyclictest0-21swapper/120:30:441
32366799322,22cyclictest465544-21ssh23:20:521
32366699323,20cyclictest12350irq/32-eth020:40:550
323666993225,5cyclictest517054-21aten_r2power_cu00:25:370
323666993222,8cyclictest334390-21/usr/sbin/munin19:45:450
323666993215,12cyclictest12350irq/32-eth000:05:550
323668993123,5cyclictest324110-21cpuspeed19:10:392
323668993123,5cyclictest2750506-21cups-browsed21:10:442
323668993121,6cyclictest338647-21munin-plugin-st20:00:012
323667993124,3cyclictest0-21swapper/119:15:011
323667993123,4cyclictest0-21swapper/123:40:591
32366799312,24cyclictest525860-21ssh00:35:541
323667993118,3cyclictest323803-21latency_hist19:10:021
32366699318,12cyclictest151ktimers/019:35:470
323666993124,4cyclictest1055-21mta-sts-daemon21:13:360
323666993121,4cyclictest1-21systemd20:15:010
323668993024,3cyclictest380363-21aten2.4-expect21:35:382
323668993024,3cyclictest380363-21aten2.4-expect21:35:382
323668993024,3cyclictest0-21swapper/220:15:002
323668993021,6cyclictest600-21gmain19:25:542
32366899301,23cyclictest497675-21ssh00:00:492
323667993022,3cyclictest0-21swapper/123:35:021
323666993023,3cyclictest0-21swapper/021:20:010
32366699302,22cyclictest389012-21ssh21:45:490
323666993020,8cyclictest446038-21unixbench_singl22:56:010
323666993019,8cyclictest261-21systemd-journal22:55:010
323666993017,10cyclictest339178-21fschecks_count20:00:430
32366899294,20cyclictest421-21dbus-daemon19:30:552
323668992920,6cyclictest421-21dbus-daemon21:10:012
323667992923,3cyclictest1056-21mta-sts-daemon19:45:591
323667992923,3cyclictest0-21swapper/121:00:001
323667992921,5cyclictest338530-21unixbench_multi19:56:001
32366699298,16cyclictest12350irq/32-eth022:00:480
32366699298,16cyclictest12350irq/32-eth022:00:480
32366699293,21cyclictest417200-21ssh22:20:490
323666992923,3cyclictest0-21swapper/021:35:000
323666992923,3cyclictest0-21swapper/021:35:000
323666992917,7cyclictest0-21swapper/021:40:490
323668992821,4cyclictest350496-21unixbench-2d20:35:592
32366799283,18cyclictest2750506-21cups-browsed20:40:491
323667992822,3cyclictest0-21swapper/119:45:011
323667992819,6cyclictest337143-21cron19:55:011
323667992817,8cyclictest339821-21switchtime20:01:001
323667992814,9cyclictest513167-21/usr/sbin/munin00:20:391
323667992814,7cyclictest171rcu_preempt19:35:541
323666992821,4cyclictest2750506-21cups-browsed21:06:190
32366999273,7cyclictest0-21swapper/321:25:473
323668992719,5cyclictest0-21swapper/221:05:002
323667992720,4cyclictest0-21swapper/120:40:011
323666992720,5cyclictest330168-21fschecks_count19:30:420
323666992718,6cyclictest12350irq/32-eth019:55:490
323666992714,10cyclictest0-21swapper/023:05:010
32366999261,8cyclictest0-21swapper/323:50:453
323667992621,2cyclictest0-21swapper/121:10:011
323667992620,3cyclictest0-21swapper/120:30:011
323667992615,2cyclictest342284-21interrupts20:10:431
323666992620,4cyclictest347862-21latency_hist20:30:020
32366699261,19cyclictest1017-21sendmail-mta19:15:480
32366899253,17cyclictest420-21avahi-daemon19:45:582
323668992519,3cyclictest349169-21kworker/u8:9-xprtiod20:50:572
323668992517,5cyclictest346168-21munin-plugin-st20:25:002
323668992517,5cyclictest346168-21munin-plugin-st20:25:002
32366799258,6cyclictest171rcu_preempt19:20:501
32366799258,6cyclictest171rcu_preempt19:20:501
32366799251,8cyclictest171rcu_preempt19:15:521
323666992521,2cyclictest0-21swapper/020:35:510
323666992520,2cyclictest0-21swapper/020:21:010
323666992520,2cyclictest0-21swapper/020:21:010
323666992516,6cyclictest324533-21ntp_kernel_pll_19:10:490
323666992513,10cyclictest0-21swapper/020:55:520
32366999241,6cyclictest0-21swapper/323:50:023
323668992416,5cyclictest328021-21unixbench-2d19:20:592
323668992416,5cyclictest328021-21unixbench-2d19:20:592
32366799247,4cyclictest171rcu_preempt20:20:501
32366799247,4cyclictest171rcu_preempt20:20:501
323666992418,4cyclictest344663-21idleruntime-cro20:20:000
32366999232,8cyclictest0-21swapper/319:30:553
32366999231,6cyclictest0-21swapper/323:20:523
323668992318,2cyclictest0-21swapper/219:10:012
323668992317,3cyclictest0-21swapper/220:07:482
32366799232,16cyclictest341008-21ntpq20:05:471
323666992317,4cyclictest2750506-21cups-browsed20:34:440
323666992316,5cyclictest358150-21cron21:05:000
323666992316,4cyclictest0-21swapper/019:40:540
32366999221,6cyclictest0-21swapper/322:30:023
32366999221,5cyclictest0-21swapper/321:55:483
32366999221,4cyclictest0-21swapper/300:00:503
323668992215,3cyclictest0-21swapper/220:30:012
323666992216,4cyclictest328160-21cron19:25:000
323666992216,4cyclictest328160-21cron19:25:000
32366999212,6cyclictest0-21swapper/300:35:403
32366999211,8cyclictest0-21swapper/322:15:473
32366999211,8cyclictest0-21swapper/300:10:013
32366999211,6cyclictest0-21swapper/323:15:503
32366999211,6cyclictest0-21swapper/323:10:513
32366999211,6cyclictest0-21swapper/322:50:473
32366899211,17cyclictest352766-21awk20:45:422
32366999203,5cyclictest0-21swapper/321:50:483
323666992015,2cyclictest0-21swapper/019:10:010
32366999194,7cyclictest0-21swapper/321:30:523
32366999194,7cyclictest0-21swapper/321:30:523
32366999194,4cyclictest0-21swapper/320:45:493
32366999192,8cyclictest0-21swapper/323:40:523
32366999183,3cyclictest0-21swapper/320:35:473
32366999181,6cyclictest0-21swapper/322:30:413
32366999174,5cyclictest0-21swapper/321:20:533
32366999171,8cyclictest0-21swapper/321:05:533
32366999164,10cyclictest0-21swapper/322:45:473
32366999164,10cyclictest0-21swapper/322:45:473
32366999162,8cyclictest0-21swapper/321:10:543
32366999162,7cyclictest0-21swapper/300:30:503
32366999162,7cyclictest0-21swapper/300:25:493
32366999161,8cyclictest0-21swapper/323:00:413
32366999161,8cyclictest0-21swapper/322:35:483
32366999152,8cyclictest0-21swapper/323:30:493
32366999152,8cyclictest0-21swapper/300:15:533
32366999152,7cyclictest0-21swapper/322:55:493
32366999151,6cyclictest0-21swapper/320:40:503
32366999143,6cyclictest0-21swapper/322:20:463
32366999142,8cyclictest0-21swapper/323:25:523
32366999142,7cyclictest0-21swapper/321:45:513
32366999141,9cyclictest0-21swapper/323:56:003
32366999141,9cyclictest0-21swapper/323:56:003
32366999141,8cyclictest0-21swapper/323:05:533
32366999133,5cyclictest0-21swapper/322:00:443
32366999133,5cyclictest0-21swapper/322:00:443
32366999132,7cyclictest0-21swapper/323:35:503
32366999132,6cyclictest0-21swapper/321:35:443
32366999132,6cyclictest0-21swapper/321:35:443
32366999123,7cyclictest0-21swapper/321:15:443
32366999123,6cyclictest0-21swapper/300:20:393
32366999122,7cyclictest0-21swapper/321:40:423
32366999121,8cyclictest0-21swapper/322:40:403
32366999121,7cyclictest0-21swapper/320:15:533
32366999113,6cyclictest0-21swapper/300:10:523
32366999112,5cyclictest0-21swapper/320:55:453
32366999111,7cyclictest0-21swapper/322:10:403
32366999104,4cyclictest0-21swapper/320:05:483
32366999104,4cyclictest0-21swapper/319:50:483
32366999102,5cyclictest0-21swapper/320:10:443
32366999101,7cyclictest0-21swapper/322:05:443
32366999101,7cyclictest0-21swapper/322:05:443
32366999101,7cyclictest0-21swapper/319:35:513
32366999101,6cyclictest0-21swapper/321:00:443
3236699993,4cyclictest0-21swapper/319:15:003
3236699991,6cyclictest0-21swapper/320:30:543
3236699991,6cyclictest0-21swapper/319:40:503
3236699991,6cyclictest0-21swapper/319:25:513
3236699991,6cyclictest0-21swapper/319:20:513
3236699991,6cyclictest0-21swapper/319:20:513
3236699991,6cyclictest0-21swapper/319:15:523
3236699991,5cyclictest0-21swapper/320:25:023
3236699991,5cyclictest0-21swapper/320:25:023
3236699991,4cyclictest0-21swapper/320:05:023
3236699983,4cyclictest0-21swapper/320:25:463
3236699983,3cyclictest0-21swapper/319:45:523
3236699982,5cyclictest0-21swapper/319:55:473
3236699981,6cyclictest0-21swapper/320:50:493
3236699970,4cyclictest0-21swapper/319:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional