You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-20 - 23:56

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot5s.osadl.org (updated Sat Dec 20, 2025 12:44:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1180992523,1cyclictest9018-21cron11:50:013
117299250,1cyclictest0-21swapper/109:35:011
1170992524,0cyclictest15212-21apt-get10:59:530
1170992523,1cyclictest32524-21sh09:20:000
117099250,24cyclictest2066-21apt-get10:30:010
1172992423,1cyclictest28690-21latency_hist08:05:011
117299240,23cyclictest606-21dbus-daemon09:15:001
1170992421,2cyclictest30943-21awk08:10:010
1180992321,1cyclictest4703-21turbostat10:35:013
1180992321,1cyclictest16877-21apt-get12:09:533
1180992321,1cyclictest0-21swapper/309:10:003
1180992319,3cyclictest3386-21latency_hist08:20:003
118099230,22cyclictest6572-21munin-run07:20:013
118099230,22cyclictest32561-21lockfile-create09:20:003
1176992321,1cyclictest856-21munin-run08:15:012
117699230,2cyclictest3890-21sh07:15:012
117699230,2cyclictest11462-21grep07:30:012
117699230,22cyclictest22672-21grep09:00:012
1172992320,2cyclictest32530-21ls09:20:001
1172992320,2cyclictest20392-21grep12:15:001
117299231,21cyclictest14619-21sed10:55:011
117299230,22cyclictest5609-21apt-get10:39:541
117299230,22cyclictest0-21swapper/111:19:541
117299230,22cyclictest0-21swapper/110:00:001
117299230,22cyclictest0-21swapper/108:54:521
117299230,1cyclictest0-21swapper/110:20:011
1170992321,1cyclictest313-21systemd-journal12:10:010
1170992320,2cyclictest606-21dbus-daemon10:50:000
117099230,22cyclictest26662-21munin-run10:15:010
117099230,22cyclictest25093-21perf09:05:010
117099230,22cyclictest19807-21apt-get07:49:510
1180992220,1cyclictest13937-21munin-run07:35:003
118099220,21cyclictest609-21rs:main3
118099220,21cyclictest31858-21apt-get08:14:533
118099220,21cyclictest2463-21apt-get11:39:533
118099220,21cyclictest1946-21awk09:25:013
118099220,21cyclictest0-21swapper/309:45:013
118099220,21cyclictest0-21swapper/309:05:013
118099220,1cyclictest19301-21sendmail10:00:003
118099220,0cyclictest0-21swapper/310:29:523
1176992220,1cyclictest9889-21munin-run09:40:012
1176992220,1cyclictest21832-21sensors11:10:012
117699220,21cyclictest6764-21perf10:40:002
117699220,21cyclictest4266-21apt-get08:24:512
117699220,21cyclictest31422-21munin-run11:30:002
117699220,21cyclictest0-21swapper/212:09:532
117699220,21cyclictest0-21swapper/210:54:522
117699220,21cyclictest0-21swapper/210:04:542
117699220,1cyclictest32549-21munin-run09:20:002
1172992220,1cyclictest21378-21perf11:10:011
1172992219,2cyclictest13896-21cron07:35:001
117299220,21cyclictest7272-21apt-get11:49:501
117299220,21cyclictest5890-21sed08:25:001
117299220,21cyclictest30036-21apt-get10:24:531
117299220,21cyclictest0-21swapper/111:44:521
117299220,21cyclictest0-21swapper/109:39:521
117299220,21cyclictest0-21swapper/109:24:541
117299220,21cyclictest0-21swapper/107:20:011
117299220,1cyclictest0-21swapper/108:20:001
1170992221,1cyclictest22204-21apt-get07:54:520
1170992220,1cyclictest32585-21latency_hist12:40:010
1170992220,1cyclictest32585-21latency_hist12:40:010
1170992220,1cyclictest28933-21apt-get12:34:560
117099220,21cyclictest10410-21latency_hist08:35:000
117099220,21cyclictest0-21swapper/011:49:510
117099220,21cyclictest0-21swapper/009:15:000
117099220,21cyclictest0-21swapper/007:24:520
1180992120,0cyclictest0-21swapper/311:09:533
1180992119,1cyclictest28820-21perf11:25:003
118099210,21cyclictest0-21swapper/311:54:523
118099210,0cyclictest8361-21apt-get09:39:523
118099210,0cyclictest0-21swapper/311:29:533
1176992121,0cyclictest0-21swapper/211:39:532
1176992120,1cyclictest17412-21apt-get07:44:522
1176992119,1cyclictest28282-21wc12:30:012
117699210,21cyclictest0-21swapper/211:34:552
117699210,20cyclictest0-21swapper/211:04:522
117699210,20cyclictest0-21swapper/210:59:532
117699210,20cyclictest0-21swapper/208:29:522
117699210,0cyclictest0-21swapper/212:14:542
117699210,0cyclictest0-21swapper/209:34:552
117699210,0cyclictest0-21swapper/207:24:512
1172992121,0cyclictest0-21swapper/109:44:521
1172992120,0cyclictest0-21swapper/112:34:561
1172992120,0cyclictest0-21swapper/111:04:521
1172992119,1cyclictest313-21systemd-journal10:10:001
1172992119,1cyclictest26414-21cron08:00:011
117299210,20cyclictest2501-21apt-get07:14:531
117299210,0cyclictest29463-21apt-get08:09:531
117299210,0cyclictest0-21swapper/111:24:541
117299210,0cyclictest0-21swapper/110:25:021
117299210,0cyclictest0-21swapper/108:34:531
1170992121,0cyclictest21373-21apt-get11:10:020
1170992121,0cyclictest11455-21apt-get08:39:530
1170992120,1cyclictest4893-21apt-get07:19:520
1170992120,1cyclictest27679-21apt-get11:24:540
1170992120,1cyclictest11434-21apt-get07:30:030
1170992120,0cyclictest1854-21apt-get08:19:530
1170992120,0cyclictest0-21swapper/009:39:520
1170992119,1cyclictest22697-21ls09:00:010
1170992118,2cyclictest15527-21latency_hist12:05:010
117099210,20cyclictest0-21swapper/012:29:510
117099210,20cyclictest0-21swapper/008:49:520
117099210,1cyclictest17964-21apt-get09:59:530
117099210,1cyclictest0-21swapper/010:39:540
117099210,0cyclictest0-21swapper/010:44:530
1180992020,0cyclictest0-21swapper/312:19:543
1180992020,0cyclictest0-21swapper/309:34:553
1180992020,0cyclictest0-21swapper/309:29:523
1180992020,0cyclictest0-21swapper/308:34:533
1180992020,0cyclictest0-21swapper/308:04:523
1180992020,0cyclictest0-21swapper/307:39:523
1180992019,1cyclictest0-21swapper/308:49:523
1180992018,1cyclictest17191-21latency_hist09:55:003
1180992018,1cyclictest13524-21munin-run12:00:013
1180992017,2cyclictest29735-21apt09:15:003
118099200,0cyclictest0-21swapper/312:29:523
118099200,0cyclictest0-21swapper/312:04:563
118099200,0cyclictest0-21swapper/310:04:533
118099200,0cyclictest0-21swapper/308:39:533
118099200,0cyclictest0-21swapper/307:29:523
1176992020,0cyclictest8025-21apt-get10:44:532
1176992020,0cyclictest0-21swapper/211:49:512
1176992020,0cyclictest0-21swapper/210:14:522
1176992020,0cyclictest0-21swapper/208:49:522
1176992018,1cyclictest27300-21turbostat09:10:002
1172992020,0cyclictest0-21swapper/111:54:521
1172992020,0cyclictest0-21swapper/110:44:531
1172992020,0cyclictest0-21swapper/110:14:531
1172992020,0cyclictest0-21swapper/109:54:511
117299200,20cyclictest0-21swapper/112:04:571
117299200,0cyclictest0-21swapper/112:29:521
117299200,0cyclictest0-21swapper/108:59:511
1170992020,0cyclictest9705-21apt-get07:29:520
1170992020,0cyclictest0-21swapper/011:45:010
1170992020,0cyclictest0-21swapper/011:32:020
1170992020,0cyclictest0-21swapper/011:04:520
1170992020,0cyclictest0-21swapper/007:59:510
1170992019,1cyclictest19271-21apt-get12:14:540
1170992019,0cyclictest27070-21apt-get08:04:520
1170992018,1cyclictest29144-21awk10:20:010
1170992018,1cyclictest0-21swapper/011:55:010
117099200,0cyclictest0-21swapper/011:09:540
117099200,0cyclictest0-21swapper/008:29:520
1180991919,0cyclictest24082-21apt-get12:24:533
1180991919,0cyclictest0-21swapper/312:34:563
1180991919,0cyclictest0-21swapper/311:19:543
1180991919,0cyclictest0-21swapper/310:49:523
1180991919,0cyclictest0-21swapper/310:39:543
1180991918,1cyclictest32477-21apt-get11:34:563
1180991917,1cyclictest16661-21munin-run11:00:013
1180991917,1cyclictest0-21swapper/307:24:523
1176991919,0cyclictest0-21swapper/212:24:532
1176991919,0cyclictest0-21swapper/210:24:532
1176991919,0cyclictest0-21swapper/209:50:012
1176991919,0cyclictest0-21swapper/208:44:512
1176991919,0cyclictest0-21swapper/208:19:532
1176991918,1cyclictest0-21swapper/210:30:022
117699190,1cyclictest9671-21apt-get11:54:512
117699190,18cyclictest0-21swapper/208:04:522
117299192,16cyclictest3173-21apt-get09:29:521
1172991919,0cyclictest0-21swapper/111:59:531
1172991919,0cyclictest0-21swapper/111:39:531
1172991919,0cyclictest0-21swapper/110:59:531
1172991919,0cyclictest0-21swapper/107:49:521
1172991917,1cyclictest0-21swapper/112:20:011
117299190,14cyclictest313-21systemd-journal09:45:011
1170991919,0cyclictest18664-21apt-get08:54:520
1170991919,0cyclictest0-21swapper/010:24:530
1170991919,0cyclictest0-21swapper/009:24:540
1170991919,0cyclictest0-21swapper/008:14:530
1170991918,1cyclictest25207-21apt-get11:19:540
1180991818,0cyclictest13867-21apt-get08:44:513
1180991818,0cyclictest0-21swapper/312:14:543
1180991818,0cyclictest0-21swapper/310:19:523
1180991818,0cyclictest0-21swapper/307:14:533
1180991815,2cyclictest8092-21awk08:30:003
1176991818,0cyclictest0-21swapper/211:14:532
1176991818,0cyclictest0-21swapper/210:19:532
1176991818,0cyclictest0-21swapper/209:44:522
1176991817,1cyclictest0-21swapper/212:05:012
117699181,16cyclictest23996-21perl09:04:542
117699180,18cyclictest0-21swapper/212:39:542
117699180,18cyclictest0-21swapper/212:39:542
117699180,17cyclictest0-21swapper/211:24:542
1172991818,0cyclictest0-21swapper/112:09:531
1172991818,0cyclictest0-21swapper/111:34:561
1170991818,0cyclictest0-21swapper/009:29:520
1170991817,1cyclictest2066-21apt-get10:30:020
1170991817,1cyclictest12068-21apt-get11:59:530
1170991817,0cyclictest22773-21apt-get10:09:520
1170991816,1cyclictest606-21dbus-daemon07:45:000
1170991816,1cyclictest20381-21apt-get10:04:540
117099180,18cyclictest5869-21perf08:25:000
1180991717,0cyclictest0-21swapper/310:14:523
1180991717,0cyclictest0-21swapper/309:49:533
1180991717,0cyclictest0-21swapper/308:54:523
1180991716,0cyclictest31326-21apt-get12:39:543
1180991716,0cyclictest31326-21apt-get12:39:543
1176991717,0cyclictest0-21swapper/212:19:552
1176991717,0cyclictest0-21swapper/210:09:522
1176991717,0cyclictest0-21swapper/207:39:522
1176991715,2cyclictest0-21swapper/209:59:532
1176991714,2cyclictest0-21swapper/209:25:022
1172991717,0cyclictest7313-21apt-get07:24:521
1172991717,0cyclictest0-21swapper/110:04:541
1172991717,0cyclictest0-21swapper/109:04:521
1172991717,0cyclictest0-21swapper/108:44:511
1172991717,0cyclictest0-21swapper/107:39:521
1172991715,1cyclictest22262-21apt-get11:10:021
1170991717,0cyclictest6882-21gpgv09:35:010
1170991717,0cyclictest0-21swapper/009:54:510
1170991717,0cyclictest0-21swapper/008:44:510
1170991716,1cyclictest14996-21apt-get07:39:520
1170991715,1cyclictest14615-21munin-run09:50:010
1170991715,1cyclictest14392-21munin-run10:55:010
1180991615,1cyclictest21055-21apt-get08:59:513
1176991616,0cyclictest0-21swapper/212:34:552
1176991616,0cyclictest0-21swapper/212:00:012
1176991616,0cyclictest0-21swapper/211:19:542
1176991616,0cyclictest0-21swapper/209:54:512
1176991616,0cyclictest0-21swapper/207:34:522
1176991615,1cyclictest28-21ksoftirqd/209:24:112
1176991615,0cyclictest0-21swapper/207:59:512
1176991614,1cyclictest10421-21apt-get10:49:522
1176991614,1cyclictest0-21swapper/210:25:022
1176991614,1cyclictest0-21swapper/209:15:002
117699160,15cyclictest6505-21cron11:45:012
1172991616,0cyclictest0-21swapper/107:29:521
1172991614,1cyclictest0-21swapper/112:39:541
1172991614,1cyclictest0-21swapper/112:39:541
117299160,15cyclictest0-21swapper/108:39:561
1170991616,0cyclictest24081-21apt12:24:530
1170991614,1cyclictest12445-21perf09:45:010
1180991515,0cyclictest0-21swapper/311:10:023
1180991515,0cyclictest0-21swapper/310:24:533
1180991515,0cyclictest0-21swapper/310:09:523
1180991515,0cyclictest0-21swapper/307:44:533
1180991513,2cyclictest622-21gdbus08:25:003
1180991513,1cyclictest6529-21awk11:45:013
1176991515,0cyclictest0-21swapper/208:34:532
1176991514,1cyclictest28-21ksoftirqd/207:45:102
118099140,14cyclictest0-21swapper/311:00:163
118099140,14cyclictest0-21swapper/307:57:583
118099140,0cyclictest0-21swapper/310:50:383
118099140,0cyclictest0-21swapper/310:40:483
117699140,14cyclictest0-21swapper/208:35:172
117699140,14cyclictest0-21swapper/208:06:422
117699140,14cyclictest0-21swapper/207:50:062
117699140,14cyclictest0-21swapper/207:15:182
1172991414,0cyclictest0-21swapper/110:30:021
1172991414,0cyclictest0-21swapper/109:05:101
1172991414,0cyclictest0-21swapper/108:45:061
1172991411,2cyclictest774-21runrttasks10:45:021
117299140,0cyclictest0-21swapper/107:50:371
1170991414,0cyclictest0-21swapper/012:15:040
1170991414,0cyclictest0-21swapper/011:35:350
1170991414,0cyclictest0-21swapper/009:05:530
117099140,14cyclictest0-21swapper/007:12:570
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional