You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-11 - 01:12

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack2slot7s.osadl.org (updated Mon Aug 10, 2026 12:43:04)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
370624995947,5cyclictest12650irq/32-eth010:45:530
370624995848,6cyclictest3791945-21cups-browsed10:05:590
370624995427,16cyclictest151ktimers/011:25:590
370624995427,16cyclictest151ktimers/011:25:590
370626995343,6cyclictest485354-21systemd-run11:20:012
370626995343,6cyclictest485354-21systemd-run11:20:012
370626995246,3cyclictest0-21swapper/210:45:572
370626994940,6cyclictest421-21dbus-daemon10:15:012
37062499481,45cyclictest70-21kswapd011:31:000
37062499481,45cyclictest70-21kswapd011:31:000
370625994725,13cyclictest421-21dbus-daemon12:10:011
370626994640,4cyclictest0-21swapper/212:40:012
370626994636,7cyclictest461239-21systemd10:40:022
370625994635,7cyclictest265-21systemd-journal10:41:061
37062599461,36cyclictest504661-21ssh11:50:581
370624994639,4cyclictest397644-21systemd-run08:40:010
370624994636,7cyclictest412795-21munin-run09:20:010
370625994536,5cyclictest443593-21aten_r2power_cu10:10:451
370624994537,5cyclictest265-21systemd-journal12:40:010
370624994529,7cyclictest12650irq/32-eth012:15:570
37062699441,39cyclictest665-21snmpd09:31:002
370625994420,14cyclictest522644-21ssh12:20:541
370624994430,11cyclictest415887-21systemd-run09:25:010
370626994335,4cyclictest0-21swapper/211:25:552
370626994335,4cyclictest0-21swapper/211:25:552
370626994333,6cyclictest420-21avahi-daemon11:21:592
370626994333,6cyclictest420-21avahi-daemon11:21:592
370625994336,4cyclictest458411-21kworker/u8:7-xprtiod11:10:531
370624994334,6cyclictest12650irq/32-eth010:25:530
370624994327,13cyclictest379660-21latency_hist07:40:020
370626994236,3cyclictest0-21swapper/208:00:002
370626994236,3cyclictest0-21swapper/208:00:002
370626994234,5cyclictest482309-21munin-plugin-st11:15:012
370626994234,5cyclictest428129-21latency_hist09:45:022
370626994234,4cyclictest0-21swapper/210:25:552
370625994234,5cyclictest415862-21munin-run09:25:011
370624994232,7cyclictest472088-21sshd10:56:190
37062499422,31cyclictest435332-21ssh09:56:020
37062599414,32cyclictest501582-21ssh11:45:551
37062599414,32cyclictest501582-21ssh11:45:551
370624994135,3cyclictest0-21swapper/012:15:010
370624994131,7cyclictest522480-21latency12:20:510
370626994034,3cyclictest0-21swapper/211:35:572
370626994019,15cyclictest406659-21diskmemload10:30:552
37062599403,27cyclictest483541-21ssh11:15:581
37062599403,27cyclictest483541-21ssh11:15:581
370625994032,5cyclictest450029-21kernelversion10:20:481
370625994032,5cyclictest434067-21cron09:55:011
370624994034,4cyclictest483437-21sh11:15:540
370624994034,4cyclictest483437-21sh11:15:540
370626993931,5cyclictest441643-21sshd10:06:092
370626993927,8cyclictest500442-21ssh11:45:012
370625993931,5cyclictest494357-21sh11:35:001
370625993931,5cyclictest494357-21sh11:35:001
370625993931,5cyclictest388633-21systemd-run08:10:011
370626993830,5cyclictest400599-21cron08:50:012
370626993830,3cyclictest0-21swapper/210:20:012
370626993829,6cyclictest3791945-21cups-browsed10:00:062
370626993827,7cyclictest388721-21latency_hist08:10:022
37062599384,26cyclictest528733-21ssh12:30:571
370625993830,5cyclictest458717-21aten_r2power_fr10:35:441
370625993830,5cyclictest411521-21head09:16:081
370625993821,13cyclictest497685-21ssh11:40:131
37062699373,28cyclictest510726-21ssh12:01:002
370626993729,5cyclictest473250-21latency_hist11:00:012
370626993727,4cyclictest0-21swapper/211:45:552
370626993727,4cyclictest0-21swapper/211:45:552
370625993730,3cyclictest0-21swapper/109:10:541
370625993730,3cyclictest0-21swapper/109:10:541
370625993729,4cyclictest0-21swapper/108:50:551
370625993727,4cyclictest0-21swapper/112:35:581
370625993714,9cyclictest3791945-21cups-browsed08:56:121
370625993714,9cyclictest3791945-21cups-browsed08:56:121
370624993729,6cyclictest439-21systemd-logind07:35:010
370624993728,6cyclictest3791945-21cups-browsed10:39:120
370624993714,19cyclictest486456-21rm11:20:550
370624993714,19cyclictest486456-21rm11:20:550
370626993628,5cyclictest387115-21sh08:05:002
370626993628,5cyclictest0-21swapper/207:40:012
370626993627,5cyclictest470230-21ssh10:55:012
370626993623,8cyclictest0-21swapper/209:16:032
370625993628,5cyclictest435487-21mailstats09:56:071
370625993628,5cyclictest390641-21fschecks_time08:15:471
370625993626,8cyclictest453427-21ntpq10:26:001
37062499364,27cyclictest0-21swapper/012:30:540
370624993631,2cyclictest0-21swapper/009:35:530
370624993630,4cyclictest500416-21grep11:45:000
370624993630,3cyclictest0-21swapper/009:50:500
370624993627,6cyclictest420-21avahi-daemon12:06:030
370624993626,7cyclictest480225-21latency11:10:480
370626993528,4cyclictest429770-21unixbench_multi09:46:062
370625993524,8cyclictest375079-21/usr/sbin/munin07:21:061
370625993523,8cyclictest470806-21aten_r2power_po10:55:431
370625993522,10cyclictest378956-21ntpq07:35:521
370624993528,4cyclictest417196-21wc09:26:000
370626993426,5cyclictest518704-21grep12:15:022
370626993425,7cyclictest476299-21sh11:05:012
37062599347,17cyclictest428-21mta-sts-daemon07:10:531
370625993428,3cyclictest0-21swapper/111:40:001
370625993427,6cyclictest400632-21systemd08:50:021
370625993426,5cyclictest443123-21systemd-run10:10:011
370625993426,5cyclictest1003-21mta-sts-daemon12:18:351
370625993425,6cyclictest420-21avahi-daemon07:40:451
370625993425,6cyclictest1001-21mta-sts-daemon09:48:021
370625993423,8cyclictest456578-21ssh10:31:071
370625993423,8cyclictest416967-21/usr/sbin/munin09:25:521
370625993423,8cyclictest405832-21/usr/sbin/munin09:05:481
370625993423,8cyclictest405832-21/usr/sbin/munin09:05:481
370624993427,4cyclictest372052-21head07:11:040
370624993426,5cyclictest524988-21/usr/sbin/munin12:26:010
370624993422,9cyclictest421-21dbus-daemon10:35:020
370626993325,6cyclictest384100-21latency_hist07:55:022
370626993325,6cyclictest384100-21latency_hist07:55:022
370626993325,5cyclictest439-21systemd-logind09:25:012
370626993325,5cyclictest373602-21cron07:20:012
370626993321,9cyclictest381933-21needreboot07:45:492
37062699331,28cyclictest665-21snmpd08:40:542
370625993325,5cyclictest392163-21fschecks_time08:20:471
370625993325,5cyclictest384126-21systemd-run07:55:011
370625993325,5cyclictest384126-21systemd-run07:55:011
370625993322,7cyclictest398161-21/usr/sbin/munin08:40:481
370625993321,9cyclictest425306-21ssh09:40:031
370625993321,10cyclictest487369-21ssh11:21:541
370625993321,10cyclictest487369-21ssh11:21:541
370624993328,2cyclictest0-21swapper/009:46:060
370624993327,4cyclictest403580-21sed08:56:090
370624993327,4cyclictest403580-21sed08:56:090
370624993327,3cyclictest0-21swapper/007:20:010
370624993324,6cyclictest12650irq/32-eth010:00:560
370624993322,9cyclictest393667-21tr08:25:470
370624993321,8cyclictest151ktimers/011:45:580
370624993321,8cyclictest151ktimers/011:45:580
370626993226,3cyclictest0-21swapper/208:25:532
370626993225,3cyclictest0-21swapper/210:45:022
370626993225,3cyclictest0-21swapper/209:55:542
370626993224,3cyclictest0-21swapper/211:50:062
37062699322,22cyclictest423212-21rm09:35:562
370625993226,4cyclictest489488-21ntpq11:25:561
370625993226,4cyclictest489488-21ntpq11:25:561
370625993226,3cyclictest465321-21ntpq10:45:531
370625993223,7cyclictest397516-21/usr/sbin/munin08:36:061
370625993223,5cyclictest1000-21mta-sts-daemon11:55:591
370625993222,7cyclictest395119-21/usr/sbin/munin08:30:471
37062599322,25cyclictest510757-21ssh12:01:011
370625993221,8cyclictest403939-21aten_r2power_en09:00:451
370625993221,8cyclictest403939-21aten_r2power_en09:00:451
370625993221,7cyclictest422383-21ssh09:35:331
370625993221,6cyclictest477636-21ssh11:06:021
370625993220,10cyclictest518443-21ssh12:14:511
370625993217,7cyclictest359131-21kworker/u8:4+xprtiod07:30:021
370624993227,2cyclictest0-21swapper/011:50:500
37062499322,25cyclictest495539-21ssh11:35:560
37062499322,25cyclictest426286-21ssh09:40:590
370624993218,12cyclictest419694-21fschecks_time09:30:480
370626993124,4cyclictest393055-21unixbench_singl08:21:102
370626993124,3cyclictest0-21swapper/212:15:512
370626993124,3cyclictest0-21swapper/209:25:582
370626993123,5cyclictest3791945-21cups-browsed08:35:062
370626993121,8cyclictest402972-21ntp_kernel_pll_08:55:572
370626993121,8cyclictest402972-21ntp_kernel_pll_08:55:572
37062699311,23cyclictest432229-21ssh09:50:562
370625993125,3cyclictest468503-21ntpq10:51:011
370625993124,4cyclictest377590-21ntpq07:30:581
370625993123,6cyclictest474309-21/usr/sbin/munin11:00:501
370625993123,6cyclictest389554-21/usr/sbin/munin08:10:571
370625993123,6cyclictest384837-21/usr/sbin/munin07:55:491
370625993123,6cyclictest384837-21/usr/sbin/munin07:55:491
370625993122,6cyclictest447933-21ssh10:16:101
37062599312,22cyclictest420086-21ssh09:30:551
370625993121,7cyclictest524784-21ssh12:25:041
370625993120,9cyclictest439076-21ssh10:01:431
37062499313,19cyclictest12650irq/32-eth011:55:550
370624993124,5cyclictest394603-21kworker/u8:8+xprtiod09:15:010
370624993124,5cyclictest394603-21kworker/u8:8+xprtiod09:15:010
370624993123,6cyclictest475025-21head11:01:060
370626993024,3cyclictest0-21swapper/207:25:012
370626993023,4cyclictest1003-21mta-sts-daemon11:55:492
370626993023,4cyclictest1002-21mta-sts-daemon12:22:062
370626993023,4cyclictest0-21swapper/210:20:472
370626993021,6cyclictest370636-21unin-run07:10:022
370625993024,4cyclictest386569-21ntpq08:00:591
370625993024,4cyclictest373028-21ntpq07:15:551
370624993022,5cyclictest439-21systemd-logind07:55:010
370624993022,5cyclictest439-21systemd-logind07:55:010
370624993016,11cyclictest0-21swapper/008:16:040
370626992922,4cyclictest0-21swapper/208:20:012
37062699292,21cyclictest513658-21rm12:05:562
370626992921,5cyclictest408369-21unixbench_multi09:11:062
370626992921,5cyclictest408369-21unixbench_multi09:11:062
370626992920,6cyclictest492039-21cpuspeed11:30:472
370626992920,6cyclictest492039-21cpuspeed11:30:472
370626992919,6cyclictest0-21swapper/212:30:562
370626992918,8cyclictest1000-21mta-sts-daemon12:25:452
370625992916,8cyclictest365326-21kworker/u8:11+rpciod07:10:031
370624992922,4cyclictest12650irq/32-eth010:16:070
370624992920,7cyclictest1000-21mta-sts-daemon10:22:100
370626992821,4cyclictest1001-21mta-sts-daemon08:52:432
370625992822,4cyclictest381855-21memory07:45:481
370625992816,10cyclictest394117-21open_files08:25:591
370624992822,4cyclictest468299-21needreboot10:50:520
370624992822,3cyclictest0-21swapper/010:40:580
370624992822,2cyclictest0-21swapper/008:20:480
370624992821,5cyclictest446119-21munin-run10:15:010
370624992821,4cyclictest382633-21systemd-run07:50:010
370624992821,4cyclictest381146-21systemd07:45:020
370624992819,6cyclictest1003-21mta-sts-daemon09:10:010
370624992819,6cyclictest1003-21mta-sts-daemon09:10:010
37062699272,18cyclictest477415-21ssh11:05:542
370626992720,3cyclictest0-21swapper/209:10:022
370626992720,3cyclictest0-21swapper/209:10:022
370626992719,5cyclictest3791945-21cups-browsed07:27:212
370626992713,9cyclictest351ktimers/207:40:562
37062499273,19cyclictest151ktimers/011:06:000
37062499272,15cyclictest385509-21sed07:56:040
37062499272,15cyclictest385509-21sed07:56:040
370624992721,3cyclictest151ktimers/008:45:590
370624992718,7cyclictest0-21swapper/008:01:060
370626992619,4cyclictest1001-21mta-sts-daemon07:15:012
370624992621,3cyclictest0-21swapper/012:00:470
370624992616,7cyclictest387965-21ntp_kernel_err08:05:530
370624992614,9cyclictest0-21swapper/008:40:480
370624992613,3cyclictest0-21swapper/007:25:010
370626992520,3cyclictest0-21swapper/208:35:012
370626992519,3cyclictest0-21swapper/207:30:562
370624992518,4cyclictest389821-21threads08:11:040
370626992417,5cyclictest389992-21tail08:11:062
370624992417,5cyclictest394944-21aten_r2power_fr08:30:450
370624992417,4cyclictest404071-21/usr/sbin/munin09:00:470
370624992417,4cyclictest404071-21/usr/sbin/munin09:00:470
370624992416,5cyclictest375654-21fschecks_time07:25:490
370626992214,5cyclictest404062-21aten_r2power_vo09:00:462
370626992214,5cyclictest404062-21aten_r2power_vo09:00:462
370624992216,3cyclictest0-21swapper/008:50:510
37062799202,8cyclictest0-21swapper/309:26:003
370624992016,2cyclictest0-21swapper/007:10:010
37062799192,10cyclictest0-21swapper/311:31:003
37062799192,10cyclictest0-21swapper/311:31:003
37062799182,8cyclictest0-21swapper/309:35:523
37062799181,8cyclictest0-21swapper/311:35:553
37062799173,7cyclictest0-21swapper/310:25:533
37062799172,9cyclictest0-21swapper/310:05:593
37062799172,9cyclictest0-21swapper/309:40:573
37062799172,8cyclictest0-21swapper/309:10:573
37062799172,8cyclictest0-21swapper/309:10:573
37062799172,7cyclictest0-21swapper/312:35:543
37062799171,9cyclictest0-21swapper/310:50:553
37062799171,7cyclictest0-21swapper/308:25:593
37062799163,8cyclictest0-21swapper/311:25:583
37062799163,8cyclictest0-21swapper/311:25:583
37062799162,8cyclictest0-21swapper/311:20:553
37062799162,8cyclictest0-21swapper/311:20:553
37062799162,8cyclictest0-21swapper/310:00:553
37062799162,7cyclictest0-21swapper/311:50:443
37062799161,8cyclictest0-21swapper/307:10:543
37062799161,10cyclictest0-21swapper/311:55:553
37062799161,10cyclictest0-21swapper/311:40:543
37062799153,7cyclictest0-21swapper/312:10:543
37062799153,7cyclictest0-21swapper/310:55:553
37062799153,7cyclictest0-21swapper/309:15:583
37062799152,8cyclictest0-21swapper/310:35:553
37062799152,6cyclictest0-21swapper/309:30:533
37062799151,9cyclictest0-21swapper/310:30:573
37062799151,9cyclictest0-21swapper/310:16:023
37062799151,9cyclictest0-21swapper/309:56:013
37062799151,8cyclictest0-21swapper/309:20:573
37062799151,7cyclictest0-21swapper/308:35:573
37062799151,6cyclictest0-21swapper/308:40:593
37062799151,6cyclictest0-21swapper/307:40:533
37062799143,7cyclictest0-21swapper/312:25:563
37062799143,7cyclictest0-21swapper/312:15:533
37062799142,7cyclictest0-21swapper/309:45:523
37062799142,7cyclictest0-21swapper/308:25:013
37062799141,9cyclictest0-21swapper/311:45:583
37062799141,9cyclictest0-21swapper/311:45:583
37062799141,9cyclictest0-21swapper/310:11:023
37062799141,8cyclictest0-21swapper/312:31:023
37062799141,8cyclictest0-21swapper/312:20:513
37062799141,6cyclictest0-21swapper/307:50:553
37062799141,6cyclictest0-21swapper/307:50:553
37062799141,6cyclictest0-21swapper/307:20:523
37062799133,6cyclictest0-21swapper/311:15:523
37062799133,6cyclictest0-21swapper/311:15:523
37062799132,8cyclictest0-21swapper/312:05:513
37062799132,7cyclictest0-21swapper/310:20:513
37062799132,7cyclictest0-21swapper/309:50:563
37062799132,6cyclictest0-21swapper/311:00:433
37062799131,7cyclictest0-21swapper/312:00:493
37062799131,7cyclictest0-21swapper/311:10:423
37062799131,7cyclictest0-21swapper/308:05:013
37062799131,5cyclictest0-21swapper/307:25:543
37062799124,6cyclictest0-21swapper/307:55:583
37062799124,6cyclictest0-21swapper/307:55:583
37062799124,5cyclictest0-21swapper/307:35:553
37062799123,6cyclictest0-21swapper/310:40:493
37062799121,8cyclictest0-21swapper/308:20:013
37062799121,8cyclictest0-21swapper/307:45:533
37062799121,7cyclictest0-21swapper/311:05:493
37062799121,7cyclictest0-21swapper/308:31:003
37062799121,7cyclictest0-21swapper/307:15:493
37062799121,6cyclictest0-21swapper/310:45:053
37062799121,5cyclictest0-21swapper/308:55:553
37062799121,5cyclictest0-21swapper/308:55:553
37062799112,5cyclictest0-21swapper/309:00:493
37062799112,5cyclictest0-21swapper/309:00:493
37062799111,7cyclictest0-21swapper/309:10:003
37062799111,7cyclictest0-21swapper/309:10:003
37062799111,7cyclictest0-21swapper/308:50:553
37062799111,7cyclictest0-21swapper/308:45:503
37062799111,6cyclictest0-21swapper/307:30:523
37062799102,5cyclictest0-21swapper/308:05:543
37062799101,6cyclictest0-21swapper/308:10:543
3706279981,5cyclictest0-21swapper/307:10:013
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional