You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-08 - 01:04

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack2slot7s.osadl.org (updated Fri Nov 07, 2025 12:39:54)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2558618996852,10cyclictest1748991-21cups-browsed09:29:071
2558619995847,7cyclictest2639345-21idleruntime-cro10:20:012
2558618995344,6cyclictest2588609-21systemd-run08:50:011
2558617995341,8cyclictest1-21systemd11:15:010
2558617995236,11cyclictest12450irq/32-eth010:55:470
255861799504,33cyclictest1748991-21cups-browsed09:04:090
2558617994936,9cyclictest2725315-21idleruntime-cro12:30:010
2558618994839,5cyclictest0-21swapper/109:59:101
2558617994838,7cyclictest422-21dbus-daemon09:55:010
2558617994831,14cyclictest2566095-21idleruntime-cro07:35:010
2558619994738,6cyclictest2622216-21ntpq09:54:032
2558619994738,5cyclictest2632892-21idleruntime-cro10:10:012
2558619994737,6cyclictest2626435-21latency_hist10:00:022
255861799471,28cyclictest2656781-21sshd10:47:070
2558617994637,6cyclictest2717700-21unixbench_singl12:19:150
2558618994530,11cyclictest2646344-21kworker/u8:3+events_unbound10:55:461
255861799451,27cyclictest2668166-21ntpq11:04:120
2558618994433,8cyclictest421-21avahi-daemon10:34:131
2558618994433,8cyclictest421-21avahi-daemon10:34:131
2558618994433,8cyclictest2667730-21sshd11:04:021
255861899443,33cyclictest2597353-21awk09:14:101
2558617994436,4cyclictest12550irq/33-eth010:29:050
2558617994431,10cyclictest2574150-21fschecks_count08:03:580
2558618994334,6cyclictest421-21avahi-daemon11:43:541
255861799439,30cyclictest2561082-21ntpq07:19:070
2558617994337,4cyclictest2558628-21systemd-run07:10:020
255861799432,35cyclictest2583588-21ntpq08:34:090
255861799432,32cyclictest2625472-21sshd09:59:070
255861799431,38cyclictest2700820-21ntpq11:54:040
2558619994234,5cyclictest2648993-21grep10:35:012
2558619994234,5cyclictest2648993-21grep10:35:012
255861899423,31cyclictest2677837-21ssh11:19:111
2558617994234,5cyclictest2609764-21switchtime09:34:150
255861799423,35cyclictest2579023-21ntpq08:19:050
2558617994231,9cyclictest2611893-21ssh09:38:540
2558617994231,8cyclictest266-21systemd-journal10:12:070
2558617994228,11cyclictest1870296-21kworker/u8:14+xprtiod07:25:010
255861799421,38cyclictest2603380-21ntpq09:24:080
255861799421,36cyclictest2597167-21ntpq09:14:050
255861799421,35cyclictest2641776-21ntpq10:24:050
255861799420,25cyclictest2614956-21ssh09:43:360
255861799420,25cyclictest2614956-21ssh09:43:360
2558619994133,5cyclictest2613493-21latency_hist09:40:022
2558618994133,5cyclictest2708345-21systemd-run12:05:011
2558618994133,5cyclictest2708345-21systemd-run12:05:011
255861899411,27cyclictest2635145-21ssh10:14:061
255861799413,31cyclictest437-21mta-sts-daemon10:05:070
2558617994130,8cyclictest2648471-21switchtime10:34:240
2558617994130,8cyclictest2648471-21switchtime10:34:240
255861799412,35cyclictest2568502-21ntpq07:44:030
255861799411,35cyclictest2720746-21ntpq12:24:040
255861799411,35cyclictest2681091-21ntpq11:24:060
255861799411,35cyclictest2661616-21ntpq10:54:040
255861799411,35cyclictest2581974-21ntpq08:29:040
255861799411,23cyclictest2606324-21awk09:29:120
2558619994031,5cyclictest1-21systemd09:00:012
255861799403,32cyclictest2707457-21ntpq12:04:130
255861799403,32cyclictest2707457-21ntpq12:04:130
255861799402,34cyclictest2575976-21ntpq08:09:050
2558617994018,19cyclictest1748952-21cupsd09:47:590
255861799401,37cyclictest2676634-21ssh11:17:460
255861799401,36cyclictest2572990-21ntpq07:59:050
255861799401,35cyclictest2671479-21ntpq11:09:090
255861799401,35cyclictest2586483-21ntpq08:44:020
255861799400,26cyclictest2690034-21ssh11:38:390
255861999390,31cyclictest2724265-21ssh12:29:032
2558618993933,3cyclictest0-21swapper/110:20:021
2558618993931,5cyclictest2594180-21sendmail_mailqu09:09:101
2558618993931,5cyclictest2566120-21systemd-run07:35:021
255861799393,30cyclictest2684247-21ntpq11:29:060
255861799391,36cyclictest2636425-21ssh10:15:400
255861799391,34cyclictest2587990-21ntpq08:49:010
255861799391,34cyclictest2571530-21ntpq07:54:050
255861799391,34cyclictest2570015-21ntpq07:49:100
255861799391,33cyclictest2710330-21ntpq12:09:040
255861799391,33cyclictest2564061-21ntpq07:29:090
255861799391,33cyclictest2559496-21ntpq07:14:080
2558619993829,6cyclictest421-21avahi-daemon07:50:012
2558619993827,7cyclictest2564608-21basename07:30:012
2558618993830,5cyclictest422-21dbus-daemon09:45:021
2558618993830,5cyclictest422-21dbus-daemon09:45:021
255861899382,30cyclictest2641785-21ssh10:24:051
2558617993830,4cyclictest0-21swapper/009:19:070
255861799382,32cyclictest2589460-21ntpq08:54:010
255861799381,34cyclictest2726581-21ssh12:32:260
255861799381,34cyclictest2695577-21ssh11:46:080
255861799381,34cyclictest2594000-21ntpq09:09:040
255861799381,34cyclictest2580527-21ntpq08:24:050
255861799381,33cyclictest2694253-21ntpq11:44:070
255861799381,33cyclictest2585093-21ntpq08:39:090
255861799381,33cyclictest2577540-21ntpq08:14:050
255861799381,32cyclictest2591023-21ntpq08:59:080
2558619993729,5cyclictest2694290-21ssh11:44:072
2558619993729,3cyclictest0-21swapper/210:42:442
2558619993728,6cyclictest2705036-21systemd-run12:00:012
2558619993728,6cyclictest2658568-21timerandwakeup10:49:162
2558618993731,4cyclictest2619945-21latency_hist09:50:011
2558618993730,3cyclictest0-21swapper/111:30:071
2558618993729,5cyclictest2690207-21head11:38:551
2558618993729,5cyclictest2577146-21fschecks_time08:13:561
2558618993728,7cyclictest2731328-21latency12:39:001
2558618993724,9cyclictest2609398-21ntp_kernel_pll_09:34:041
255861799371,33cyclictest2567057-21ntpq07:39:060
255861799371,32cyclictest2732261-21ssh12:39:320
255861799371,31cyclictest2713713-21ntpq12:14:060
255861799371,28cyclictest2626942-21ssh10:01:180
255861799371,19cyclictest2594597-21diskmemload11:58:190
255861799370,35cyclictest2652898-21ssh10:40:400
2558619993628,5cyclictest2665725-21plymouth11:00:012
2558619993619,14cyclictest351ktimers/207:54:062
2558618993629,4cyclictest2683536-21/usr/sbin/munin11:29:061
2558618993627,6cyclictest266-21systemd-journal11:10:011
255861899362,26cyclictest2603321-21ssh09:24:061
255861799361,31cyclictest2686254-21ssh11:32:290
2558619993527,5cyclictest2652373-21sh10:40:012
2558619993527,5cyclictest2561604-21cut07:20:012
2558618993529,3cyclictest0-21swapper/111:54:091
2558618993528,4cyclictest2716904-21missed_timers12:19:001
2558618993527,5cyclictest2560120-21systemd-run07:15:021
2558618993527,5cyclictest2558622-21systemd-run07:10:021
2558618993527,5cyclictest1748991-21cups-browsed11:11:071
255861799350,33cyclictest2649508-21ssh10:35:470
2558619993426,5cyclictest2721387-21grep12:24:162
2558619993425,7cyclictest2592435-21needreboot09:04:022
2558619993424,7cyclictest2717710-21unixbench_singl12:19:152
2558618993429,2cyclictest0-21swapper/111:24:061
2558618993427,4cyclictest2724928-21head12:29:171
2558618993427,4cyclictest2698389-21latency_hist11:50:021
2558618993417,9cyclictest956-21mta-sts-daemon10:29:071
2558619993326,4cyclictest2701402-21unixbench_multi11:54:172
2558619993326,4cyclictest0-21swapper/209:25:012
2558619993325,6cyclictest0-21swapper/210:29:162
2558619993325,5cyclictest422-21dbus-daemon10:04:032
255861999331,26cyclictest2684296-21ssh11:29:082
2558618993327,3cyclictest2639350-21kworker/u8:16-xprtiod10:55:011
2558618993327,3cyclictest0-21swapper/111:59:131
2558618993326,4cyclictest2646344-21kworker/u8:3-xprtiod10:50:011
2558618993325,4cyclictest0-21swapper/112:25:011
2558618993323,7cyclictest2567594-21sendmail07:40:011
2558619993228,3cyclictest2558616-21cyclictest10:23:572
2558619993226,3cyclictest0-21swapper/209:19:092
2558619993224,5cyclictest2682021-21idleruntime-cro11:25:012
2558619993224,5cyclictest2588550-21unixbench_singl08:49:142
2558619993222,7cyclictest1748991-21cups-browsed09:40:572
2558619993222,7cyclictest1748991-21cups-browsed09:40:572
2558619993222,6cyclictest1748991-21cups-browsed11:13:262
255861999321,25cyclictest2606229-21ssh09:29:082
2558618993228,2cyclictest0-21swapper/110:08:581
2558618993226,3cyclictest2659124-21kworker/u8:6-rpciod12:10:031
2558618993226,3cyclictest0-21swapper/109:54:031
2558618993225,4cyclictest2585630-21kworker/u8:10-xprtiod09:00:011
2558618993224,5cyclictest2629707-21latency_hist10:05:021
2558618993224,5cyclictest2580995-21unixbench_multi08:24:151
255861899321,22cyclictest2612588-21ssh09:39:061
255861999315,18cyclictest2710420-21sshd12:09:072
2558619993125,3cyclictest0-21swapper/208:20:012
2558619993124,4cyclictest2635770-21ssh10:14:302
2558619993124,4cyclictest2589123-21fschecks_count08:53:562
2558619993123,5cyclictest2732677-21latency_hist12:40:032
2558619993123,5cyclictest2609833-21timerandwakeup09:34:162
2558619993122,6cyclictest2563105-21cron07:25:002
2558619993122,5cyclictest0-21swapper/211:39:032
2558618993125,3cyclictest0-21swapper/110:45:011
2558618993123,5cyclictest2651034-21fschecks_time10:38:591
2558619993023,4cyclictest2093187-21kworker/u8:1+rpciod08:35:012
2558619993022,5cyclictest2662578-21munin-plugin-st10:55:012
2558619993022,5cyclictest2573622-21systemd-run08:00:012
2558619993022,5cyclictest2566090-21cron07:35:012
2558619993021,6cyclictest2671599-21processes11:09:122
255861999301,24cyclictest565-21gmain11:34:022
2558618993019,6cyclictest0-21swapper/108:29:071
2558619992921,5cyclictest2727282-21aten_r2power_cu12:33:542
2558619992920,6cyclictest2668289-21wc11:04:162
2558619992918,8cyclictest2677544-21latency11:19:032
2558618992924,3cyclictest2558616-21cyclictest12:10:011
2558618992923,3cyclictest0-21swapper/112:34:161
2558618992923,3cyclictest0-21swapper/109:18:531
2558618992923,3cyclictest0-21swapper/107:20:011
2558618992922,4cyclictest2593010-21unixbench_multi09:04:161
2558618992920,6cyclictest266-21systemd-journal08:10:011
2558619992821,3cyclictest0-21swapper/209:15:022
2558619992821,3cyclictest0-21swapper/207:15:012
2558619992820,5cyclictest957-21mta-sts-daemon12:03:562
2558619992820,5cyclictest957-21mta-sts-daemon12:03:562
2558619992820,5cyclictest2581048-21unixbench_singl08:24:152
2558619992820,4cyclictest0-21swapper/208:44:082
2558619992819,5cyclictest0-21swapper/207:39:032
2558619992817,8cyclictest2619135-21ntp_kernel_pll_09:49:022
2558618992821,4cyclictest2587091-21cron08:45:011
2558618992821,4cyclictest2093187-21kworker/u8:1-xprtiod08:05:011
2558618992820,5cyclictest453-21systemd-logind07:55:021
255861899281,22cyclictest1748991-21cups-browsed07:44:091
2558619992721,2cyclictest0-21swapper/211:49:002
2558619992720,4cyclictest2594514-21unixbench_multi09:09:152
2558619992720,4cyclictest2576472-21unixbench-2d08:09:172
2558618992720,4cyclictest2093187-21kworker/u8:1-xprtiod08:00:011
2558619992619,4cyclictest956-21mta-sts-daemon12:11:052
2558619992617,7cyclictest2568906-21timerwakeupswit07:44:142
2558618992619,4cyclictest2589134-21fschecks_time08:53:561
2558618992619,4cyclictest0-21swapper/108:35:011
2558618992618,5cyclictest2585590-21munin-run08:40:011
2558619992518,3cyclictest0-21swapper/208:04:062
2558618992518,4cyclictest2578639-21grep08:18:561
2558618992518,4cyclictest2564340-21threads07:29:171
2558619992416,6cyclictest491-21gdbus08:11:542
2558619992316,4cyclictest2585502-21tail08:39:152
2558619992212,7cyclictest2581811-21latency08:29:012
2558618992217,2cyclictest0-21swapper/107:25:021
2558618992211,8cyclictest2570617-21sh07:50:011
2558619992015,2cyclictest0-21swapper/207:10:032
255862099191,9cyclictest0-21swapper/310:49:023
255862099172,8cyclictest0-21swapper/311:49:093
255862099172,7cyclictest0-21swapper/311:14:073
255862099161,7cyclictest0-21swapper/311:59:053
255862099152,7cyclictest0-21swapper/312:34:063
255862099152,7cyclictest0-21swapper/311:34:033
255862099152,7cyclictest0-21swapper/309:59:073
255862099151,7cyclictest0-21swapper/312:09:073
255862099146,5cyclictest0-21swapper/310:54:133
255862099144,5cyclictest0-21swapper/311:29:063
255862099142,8cyclictest0-21swapper/312:03:013
255862099142,8cyclictest0-21swapper/312:03:013
255862099141,8cyclictest0-21swapper/309:39:073
255862099141,7cyclictest0-21swapper/312:39:063
255862099141,7cyclictest0-21swapper/311:24:053
255862099141,7cyclictest0-21swapper/310:14:073
255862099141,7cyclictest0-21swapper/309:24:043
255862099141,7cyclictest0-21swapper/309:14:023
255862099141,6cyclictest0-21swapper/309:29:083
255862099141,6cyclictest0-21swapper/307:59:063
255862099136,5cyclictest0-21swapper/312:20:183
255862099136,4cyclictest0-21swapper/310:23:583
255862099133,4cyclictest0-21swapper/310:55:473
255862099132,7cyclictest0-21swapper/311:09:053
255862099132,7cyclictest0-21swapper/310:04:073
255862099132,6cyclictest0-21swapper/310:19:073
255862099131,8cyclictest0-21swapper/309:44:043
255862099131,8cyclictest0-21swapper/309:44:043
255862099131,7cyclictest0-21swapper/312:14:033
255862099131,7cyclictest0-21swapper/311:54:033
255862099131,7cyclictest0-21swapper/311:44:033
255862099131,7cyclictest0-21swapper/309:34:023
255862099131,6cyclictest0-21swapper/309:48:503
255862099131,6cyclictest0-21swapper/308:14:033
255862099131,6cyclictest0-21swapper/307:39:023
255862099131,6cyclictest0-21swapper/307:34:083
255862099131,6cyclictest0-21swapper/307:24:043
255862099131,5cyclictest0-21swapper/308:34:043
255862099126,4cyclictest0-21swapper/308:19:143
255862099126,4cyclictest0-21swapper/307:44:033
255862099125,4cyclictest0-21swapper/310:39:363
255862099125,4cyclictest0-21swapper/308:04:093
255862099122,6cyclictest0-21swapper/308:54:113
255862099121,7cyclictest0-21swapper/312:28:543
255862099121,7cyclictest0-21swapper/310:44:063
255862099121,6cyclictest0-21swapper/312:19:013
255862099121,6cyclictest0-21swapper/310:35:033
255862099121,6cyclictest0-21swapper/310:35:033
255862099121,6cyclictest0-21swapper/310:25:073
255862099121,6cyclictest0-21swapper/309:54:033
255862099121,6cyclictest0-21swapper/309:19:053
255862099121,6cyclictest0-21swapper/309:04:073
255862099121,5cyclictest0-21swapper/309:09:033
255862099121,5cyclictest0-21swapper/308:49:003
255862099121,5cyclictest0-21swapper/307:19:043
255862099116,3cyclictest0-21swapper/311:00:413
255862099116,3cyclictest0-21swapper/307:53:593
255862099115,4cyclictest0-21swapper/311:37:243
255862099115,4cyclictest0-21swapper/308:24:093
255862099111,6cyclictest0-21swapper/308:44:073
255862099111,5cyclictest0-21swapper/307:49:103
255862099105,4cyclictest0-21swapper/307:13:553
255862099105,3cyclictest0-21swapper/308:28:563
255862099105,3cyclictest0-21swapper/308:09:123
255862099104,4cyclictest0-21swapper/308:39:073
255862099101,6cyclictest0-21swapper/311:19:063
255862099101,6cyclictest0-21swapper/310:08:533
25586209995,3cyclictest0-21swapper/308:59:053
25586209995,3cyclictest0-21swapper/307:29:023
25586209981,4cyclictest0-21swapper/307:10:033
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional