You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-19 - 04:21

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackeslot5s.osadl.org (updated Wed Nov 19, 2025 00:44:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2003199250,2cyclictest365-21awk19:35:003
20024992523,1cyclictest593-21dbus-daemon22:05:001
2002099250,24cyclictest27595-21cron19:25:000
2003199240,1cyclictest9778-21idleruntime-cro22:05:003
2002999240,22cyclictest818-40gzip00:00:012
2002099240,23cyclictest0-21swapper/023:21:540
2003199230,22cyclictest23522-21sendmail-msp23:40:003
2002999230,22cyclictest8091-21awk20:54:592
2002999230,22cyclictest30438-21idleruntime-cro19:29:592
2002999230,22cyclictest17655-21apt-get00:34:592
20024992320,2cyclictest390-21latency_hist19:35:001
2002499230,1cyclictest23525-21awk23:40:001
2002499230,0cyclictest0-21swapper/121:46:541
2002499230,0cyclictest0-21swapper/119:46:541
2002499230,0cyclictest0-21swapper/100:36:541
20020992323,0cyclictest0-21swapper/020:46:560
20031992222,0cyclictest0-21swapper/321:46:533
20031992222,0cyclictest0-21swapper/300:36:543
2003199220,21cyclictest260-21jbd2/nvme0n1p2-00:00:013
2003199220,21cyclictest13268-21apt-get00:26:553
2003199220,21cyclictest0-21swapper/319:41:503
2002999220,21cyclictest25675-21dpkg20:25:002
2002999220,21cyclictest12597-21apt-get22:11:532
2002999220,21cyclictest10465-21apt00:20:002
2002999220,21cyclictest0-21swapper/223:26:532
2002999220,21cyclictest0-21swapper/220:45:012
2002999220,21cyclictest0-21swapper/220:11:522
2002999220,1cyclictest0-21swapper/219:45:032
2002499220,21cyclictest3658-21apt-get00:06:561
2002499220,21cyclictest26645-21awk22:40:001
2002499220,21cyclictest1872-21apt-get22:56:531
2002499220,21cyclictest17029-21awk22:20:011
2002499220,21cyclictest13268-21apt-get00:26:561
2002499220,21cyclictest0-21swapper/122:06:531
2002499220,21cyclictest0-21swapper/121:56:551
2002499220,21cyclictest0-21swapper/119:21:561
2002499220,21cyclictest0-21swapper/119:11:541
2002499220,1cyclictest0-21swapper/120:51:531
2002099220,22cyclictest0-21swapper/021:31:530
2002099220,21cyclictest16047-21apt-get20:06:530
2002099220,1cyclictest14600-21grep22:14:590
20031992121,0cyclictest8900-21apt-get20:56:513
20031992121,0cyclictest13651-21apt-get20:01:563
20031992121,0cyclictest0-21swapper/323:01:543
20031992121,0cyclictest0-21swapper/321:16:543
20031992121,0cyclictest0-21swapper/321:06:543
20031992121,0cyclictest0-21swapper/319:46:533
20031992120,1cyclictest331rcuc/323:00:003
20031992120,1cyclictest10201-21apt-get22:06:533
20031992120,0cyclictest0-21swapper/319:11:543
2003199210,2cyclictest25893-21awk23:44:593
2003199210,20cyclictest900-21perf20:40:013
2003199210,20cyclictest0-21swapper/322:21:553
2003199210,1cyclictest27597-21apt-get19:25:013
2003199210,0cyclictest0-21swapper/300:01:543
20029992120,1cyclictest0-21swapper/221:06:542
20029992120,0cyclictest27035-21apt-get22:41:552
20029992120,0cyclictest11230-21apt-get19:56:532
20029992120,0cyclictest0-21swapper/220:31:562
20029992120,0cyclictest0-21swapper/200:36:542
2002999210,20cyclictest20522-21apt21:20:002
2002999210,20cyclictest0-21swapper/219:11:542
2002999210,0cyclictest0-21swapper/223:36:542
20024992120,1cyclictest0-21swapper/123:42:011
20024992120,1cyclictest0-21swapper/120:36:521
20024992120,1cyclictest0-21swapper/120:06:541
20024992119,1cyclictest0-21swapper/121:20:001
2002499210,21cyclictest0-21swapper/122:21:541
2002499210,20cyclictest4686-21apt-get23:01:541
2002499210,1cyclictest0-21swapper/120:45:011
2002499210,0cyclictest0-21swapper/122:51:541
2002499210,0cyclictest0-21swapper/120:26:521
20020992121,0cyclictest0-21swapper/020:21:530
20020992120,1cyclictest22942-21perf21:24:590
20020992119,1cyclictest596-21in:imuxsock21:40:000
20020992119,1cyclictest26315-21apt-get23:46:560
2002099210,20cyclictest6437-21apt-get19:46:540
2002099210,20cyclictest3649-21apt-get19:41:520
2002099210,0cyclictest0-21swapper/023:42:010
2002099210,0cyclictest0-21swapper/000:26:560
20031992020,0cyclictest6053-21apt-get00:11:553
20031992020,0cyclictest5389-21apt-get21:56:553
20031992020,0cyclictest31883-21apt-get22:51:543
20031992020,0cyclictest26454-21apt-get20:26:523
20031992020,0cyclictest0-21swapper/322:41:533
20031992020,0cyclictest0-21swapper/322:20:013
20031992020,0cyclictest0-21swapper/321:51:533
20031992020,0cyclictest0-21swapper/321:26:543
20031992020,0cyclictest0-21swapper/319:56:533
20031992019,1cyclictest28134-21apt-get21:36:553
20031992019,0cyclictest0-21swapper/300:16:533
2003199200,20cyclictest3324-21munin-run20:45:013
2003199200,1cyclictest22948-21munin-run21:25:003
2003199200,0cyclictest11319-21apt-get21:01:533
2003199200,0cyclictest0-21swapper/321:41:523
2003199200,0cyclictest0-21swapper/321:11:573
20029992020,0cyclictest0-21swapper/223:42:012
20029992020,0cyclictest0-21swapper/223:16:562
20029992020,0cyclictest0-21swapper/222:51:542
20029992020,0cyclictest0-21swapper/222:26:532
20029992020,0cyclictest0-21swapper/222:16:552
20029992020,0cyclictest0-21swapper/222:06:542
20029992020,0cyclictest0-21swapper/221:51:532
20029992020,0cyclictest0-21swapper/220:55:022
20029992020,0cyclictest0-21swapper/220:26:532
20029992020,0cyclictest0-21swapper/219:20:012
20029992019,1cyclictest0-21swapper/219:51:542
20029992018,1cyclictest13302-21cron21:05:002
2002999200,0cyclictest0-21swapper/222:21:552
20024992020,0cyclictest25734-21apt-get21:31:531
20024992020,0cyclictest0-21swapper/123:26:541
20024992020,0cyclictest0-21swapper/122:41:541
20024992020,0cyclictest0-21swapper/122:26:521
20024992020,0cyclictest0-21swapper/121:41:531
20024992020,0cyclictest0-21swapper/119:26:531
20024992020,0cyclictest0-21swapper/100:11:551
20024992018,1cyclictest5648-21wc19:45:001
20024992018,1cyclictest24795-21perf19:20:001
2002499200,20cyclictest0-21swapper/123:21:531
2002499200,0cyclictest0-21swapper/121:36:551
2002499200,0cyclictest0-21swapper/100:16:531
20020992020,0cyclictest20392-21apt-get19:11:530
20020992020,0cyclictest0-21swapper/022:41:540
20020992020,0cyclictest0-21swapper/022:26:530
20020992020,0cyclictest0-21swapper/022:16:550
20020992020,0cyclictest0-21swapper/019:56:540
20020992020,0cyclictest0-21swapper/000:11:550
20020992018,1cyclictest25322-21dump-pmu-power21:30:010
2002099200,20cyclictest0-21swapper/023:31:550
2002099200,0cyclictest20448-21apt-get20:15:010
2002099200,0cyclictest0-21swapper/023:06:530
2002099200,0cyclictest0-21swapper/020:51:530
2002099200,0cyclictest0-21swapper/020:01:540
2002099200,0cyclictest0-21swapper/000:05:000
20031991919,0cyclictest16692-21apt-get23:26:543
20031991919,0cyclictest0-21swapper/323:06:543
20031991919,0cyclictest0-21swapper/300:06:563
20031991918,1cyclictest9480-21apt-get23:11:553
20029991919,0cyclictest3876-21apt-get23:00:022
20029991919,0cyclictest16119-21apt-get21:11:572
20029991919,0cyclictest0-21swapper/222:46:542
20029991919,0cyclictest0-21swapper/221:31:522
20029991919,0cyclictest0-21swapper/219:16:532
20029991918,1cyclictest0-21swapper/220:36:532
20029991918,0cyclictest14296-21apt-get23:21:532
20029991917,1cyclictest21082-21awk23:34:592
20024991919,0cyclictest0-21swapper/123:06:551
20024991919,0cyclictest0-21swapper/121:01:531
20024991919,0cyclictest0-21swapper/100:21:521
20024991918,0cyclictest0-21swapper/122:31:561
20020991919,0cyclictest17416-21apt-get22:21:550
20020991919,0cyclictest0-21swapper/023:26:540
20020991919,0cyclictest0-21swapper/021:01:530
20020991919,0cyclictest0-21swapper/020:55:020
20020991919,0cyclictest0-21swapper/019:51:540
20020991919,0cyclictest0-21swapper/000:31:540
20031991818,0cyclictest0-21swapper/323:21:543
20031991818,0cyclictest0-21swapper/322:11:533
20031991818,0cyclictest0-21swapper/320:16:553
20031991818,0cyclictest0-21swapper/319:20:013
20031991817,0cyclictest0-21swapper/320:06:543
20029991818,0cyclictest30603-21apt-get21:41:532
20029991818,0cyclictest0-21swapper/222:56:532
20029991818,0cyclictest0-21swapper/221:26:532
20029991816,1cyclictest5623-21cron00:10:002
20029991815,2cyclictest28324-21munin-run23:50:002
20024991818,0cyclictest0-21swapper/122:11:531
20024991818,0cyclictest0-21swapper/119:36:551
2002499180,0cyclictest30676-21kworker/1:220:16:551
20020991818,0cyclictest31164-21apt-get23:56:550
20020991818,0cyclictest0-21swapper/022:56:530
20020991818,0cyclictest0-21swapper/022:51:540
20020991818,0cyclictest0-21swapper/020:41:560
20020991818,0cyclictest0-21swapper/019:36:540
20020991816,1cyclictest4980-21latency_hist21:55:000
20020991815,2cyclictest6677-21apt23:04:590
20031991717,0cyclictest0-21swapper/322:26:533
20031991717,0cyclictest0-21swapper/321:31:533
20031991716,1cyclictest24827-21sendmail-msp19:20:003
20029991717,0cyclictest28709-21apt-get23:51:542
20029991717,0cyclictest0-21swapper/222:36:542
20029991717,0cyclictest0-21swapper/222:01:552
20029991717,0cyclictest0-21swapper/221:21:542
20029991717,0cyclictest0-21swapper/219:41:522
20029991717,0cyclictest0-21swapper/219:36:552
20029991716,0cyclictest0-21swapper/223:11:552
20029991715,1cyclictest0-21swapper/221:39:592
20024991717,0cyclictest0-21swapper/123:51:541
20024991717,0cyclictest0-21swapper/120:56:521
20024991717,0cyclictest0-21swapper/119:51:541
20024991716,1cyclictest22-21ksoftirqd/120:25:001
20024991715,1cyclictest822-21basename00:00:011
2002499170,2cyclictest151rcu_preempt00:31:591
20020991717,0cyclictest8448-21apt-get00:16:530
20020991717,0cyclictest30856-21apt-get19:31:590
20020991717,0cyclictest0-21swapper/019:26:540
20020991716,1cyclictest0-21swapper/022:00:000
2002099170,16cyclictest0-21swapper/019:16:530
2002099170,0cyclictest0-21swapper/022:36:550
20031991616,0cyclictest826-21apt-get19:36:543
20031991616,0cyclictest10868-21apt-get00:21:533
20031991615,1cyclictest22221-21apt-get22:31:563
2003199160,1cyclictest151rcu_preempt20:51:593
2003199160,14cyclictest0-21swapper/323:33:523
20029991616,0cyclictest0-21swapper/220:16:552
20029991616,0cyclictest0-21swapper/219:31:592
20029991615,0cyclictest0-21swapper/220:06:542
20024991616,0cyclictest0-21swapper/121:26:521
20024991614,1cyclictest5000-21munin-run21:55:001
2002499160,1cyclictest18440-21apt-get20:11:521
20020991616,0cyclictest7806-21apt-get22:01:540
20020991616,0cyclictest18511-21apt-get21:16:530
20020991616,0cyclictest0-21swapper/022:06:520
20020991616,0cyclictest0-21swapper/000:36:530
20020991614,1cyclictest23512-21perf23:40:000
2002099160,15cyclictest19485-21ntp_states20:11:570
20031991513,1cyclictest313-21systemd-journal19:55:003
20029991515,0cyclictest0-21swapper/221:46:542
20024991514,1cyclictest3714-21apt-get20:46:561
20024991513,1cyclictest0-21swapper/123:12:001
20024991513,1cyclictest0-21swapper/121:11:571
2002499150,14cyclictest0-21swapper/100:02:301
20020991515,0cyclictest0-21swapper/022:31:560
20020991515,0cyclictest0-21swapper/021:45:000
20020991515,0cyclictest0-21swapper/021:11:560
20020991515,0cyclictest0-21swapper/020:31:560
2002099150,5cyclictest593-21dbus-daemon22:50:000
20031991414,0cyclictest0-21swapper/323:51:543
20031991414,0cyclictest0-21swapper/320:11:523
2003199140,5cyclictest0-21swapper/323:16:563
2003199140,14cyclictest0-21swapper/323:45:133
2003199140,14cyclictest0-21swapper/322:47:503
2003199140,14cyclictest0-21swapper/322:36:013
2003199140,14cyclictest0-21swapper/320:46:183
2003199140,14cyclictest0-21swapper/320:30:593
2003199140,14cyclictest0-21swapper/320:21:503
2003199140,0cyclictest0-21swapper/300:30:013
20029991414,0cyclictest0-21swapper/222:30:472
2002999140,14cyclictest0-21swapper/223:05:102
2002999140,14cyclictest0-21swapper/220:45:482
2002999140,14cyclictest0-21swapper/200:20:082
2002999140,14cyclictest0-21swapper/200:11:492
2002999140,0cyclictest0-21swapper/200:29:242
2002999140,0cyclictest0-21swapper/200:01:402
20024991414,0cyclictest0-21swapper/123:31:381
2002499140,14cyclictest0-21swapper/123:46:001
2002499140,14cyclictest0-21swapper/123:16:231
2002499140,14cyclictest0-21swapper/121:06:471
2002499140,14cyclictest0-21swapper/119:55:461
2002499140,0cyclictest0-21swapper/120:31:391
20020991414,0cyclictest0-21swapper/021:45:480
20020991414,0cyclictest0-21swapper/021:05:140
20020991414,0cyclictest0-21swapper/020:26:510
20020991414,0cyclictest0-21swapper/000:21:520
20020991414,0cyclictest0-21swapper/000:07:050
20020991413,1cyclictest31306-21apt-get20:36:530
2002099140,14cyclictest0-21swapper/023:15:420
2002099140,14cyclictest0-21swapper/023:11:400
20029991311,1cyclictest0-21swapper/220:01:562
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional