You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-12-05 - 15:29

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack2slot7s.osadl.org (updated Fri Dec 05, 2025 12:39:38)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
134640995741,12cyclictest422-21dbus-daemon10:10:012
134639995544,8cyclictest240336-21gltestperf11:24:141
134640995447,3cyclictest0-21swapper/209:45:002
134640995447,3cyclictest0-21swapper/209:45:002
134638995236,13cyclictest136142-21systemd-run07:15:010
134638995236,13cyclictest136142-21systemd-run07:15:010
134638995040,4cyclictest0-21swapper/011:34:210
134638994937,9cyclictest160498-21timerwakeupswit09:24:310
134638994935,6cyclictest12450irq/32-eth011:14:330
134639994839,6cyclictest422-21dbus-daemon11:45:011
134639994837,7cyclictest195184-21munin-plugin-st10:15:011
13464099472,39cyclictest225027-21ssh10:59:482
134639994738,6cyclictest266-21systemd-journal12:00:011
134638994711,31cyclictest184219-21ssh09:59:220
134640994537,5cyclictest282912-21systemd-run12:25:012
134639994537,5cyclictest185152-21latency_hist10:00:021
134638994535,7cyclictest251631-21grep11:40:010
13463899452,35cyclictest268152-21ssh12:04:190
134639994437,3cyclictest0-21swapper/109:29:231
134639994436,5cyclictest290313-21latency_hist12:35:021
134638994437,4cyclictest236944-21cpuspeed11:19:120
134638994432,9cyclictest171091-21kworker/u8:15+xprtiod10:15:010
134640994335,5cyclictest255269-21idleruntime-cro11:45:012
134640994335,5cyclictest181439-21cron09:55:012
134640994334,5cyclictest161026-21latency_hist09:25:032
134638994332,9cyclictest232180-21munin-run11:10:020
134638994322,13cyclictest12550irq/33-eth011:59:180
134640994234,5cyclictest1-21systemd12:30:022
134640994210,27cyclictest250745-21ssh11:39:182
134639994232,6cyclictest286592-21true12:30:011
134639994231,8cyclictest157591-21sendmail09:20:011
134638994230,9cyclictest281729-21rm12:24:150
134638994229,10cyclictest0-21swapper/009:55:010
13464099413,30cyclictest194404-21ssh10:14:222
13463899411,36cyclictest293238-21ssh12:39:240
134639994034,3cyclictest0-21swapper/110:54:201
134639994032,5cyclictest257620-21/usr/sbin/munin11:49:141
134639994032,5cyclictest154110-21debian-sa109:15:011
134639994032,5cyclictest146826-21sendmail08:40:011
13463999402,31cyclictest218272-21ssh10:49:201
134639994017,10cyclictest150882-21diskmemload11:29:211
13464099399,22cyclictest204815-21sshd10:29:262
134640993931,5cyclictest959-21mta-sts-daemon12:33:592
134640993931,5cyclictest279129-21munin-plugin-st12:20:012
134640993931,5cyclictest136430-21systemd-run07:20:012
13463999396,25cyclictest293048-21ssh12:39:171
134639993931,5cyclictest236824-21aten_r2power_po11:19:111
134639993929,6cyclictest3499535-21cups-browsed12:03:451
13463899393,8cyclictest197896-21ssh10:19:220
134638993927,6cyclictest163670-21sh09:29:270
134640993830,5cyclictest188410-21munin-plugin-st10:05:012
134640993828,4cyclictest0-21swapper/209:59:232
134640993826,8cyclictest3499535-21cups-browsed10:43:262
134638993833,2cyclictest0-21swapper/011:50:010
134638993832,3cyclictest171079-21basename09:40:010
134640993731,3cyclictest0-21swapper/212:00:012
134640993730,4cyclictest1-21systemd11:20:022
134640993729,5cyclictest222222-21unixbench_multi10:54:322
134640993729,5cyclictest171056-21ls09:40:002
134640993729,5cyclictest148345-21systemd-run08:45:022
134640993728,6cyclictest178156-21grep09:50:002
134640993727,7cyclictest233596-21aten_r2power_fr11:14:172
13463999373,27cyclictest271368-21ssh12:09:251
134639993729,5cyclictest146291-21munin-plugin-st08:30:011
134639993729,4cyclictest0-21swapper/111:34:221
134639993728,6cyclictest437-21mta-sts-daemon09:44:221
134639993728,6cyclictest437-21mta-sts-daemon09:44:221
13463999371,3cyclictest0-21swapper/110:24:221
134638993731,4cyclictest1-21systemd11:45:010
134638993730,4cyclictest957-21mta-sts-daemon12:29:220
134638993725,8cyclictest205697-21wc10:30:020
134640993630,3cyclictest0-21swapper/209:20:012
134640993625,8cyclictest271695-21timerandwakeup12:09:322
134639993628,5cyclictest232169-21ssh11:10:021
134639993628,5cyclictest208939-21idleruntime-cro10:35:011
134639993628,4cyclictest0-21swapper/110:44:171
134639993627,5cyclictest0-21swapper/111:40:011
13463999362,25cyclictest170320-21ssh09:39:221
134638993631,2cyclictest0-21swapper/009:44:130
134638993631,2cyclictest0-21swapper/009:44:130
134638993628,5cyclictest3499511-21cupsd12:19:180
13464099352,24cyclictest243997-21ssh11:29:242
134640993521,11cyclictest247657-21switchtime11:34:322
134640993517,9cyclictest150882-21diskmemload11:54:252
13463999353,23cyclictest266-21systemd-journal09:24:261
134639993525,4cyclictest0-21swapper/110:04:231
134639993523,3cyclictest171rcu_preempt08:14:201
134638993527,5cyclictest266-21systemd-journal09:50:010
134638993521,5cyclictest272126-21sh12:10:010
134640993427,4cyclictest198714-21grep10:20:012
134640993426,5cyclictest146321-21latency_hist08:30:022
134640993426,4cyclictest0-21swapper/211:24:192
134640993425,4cyclictest0-21swapper/207:29:202
134639993427,4cyclictest0-21swapper/112:25:021
134639993426,5cyclictest0-21swapper/107:20:011
134639993423,5cyclictest0-21swapper/112:19:211
134638993426,5cyclictest212250-21cron10:40:010
134638993426,5cyclictest212250-21cron10:40:010
134638993424,7cyclictest191852-21latency_hist10:10:020
134638993418,13cyclictest289788-21threads12:34:320
134640993327,3cyclictest0-21swapper/212:14:272
134640993325,5cyclictest144544-21idleruntime-cro08:20:012
13463999336,19cyclictest177295-21ssh09:49:201
134639993326,4cyclictest422-21dbus-daemon07:15:011
134639993326,4cyclictest422-21dbus-daemon07:15:011
134639993325,3cyclictest0-21swapper/110:59:431
13463999332,2cyclictest171rcu_preempt07:59:211
134639993321,9cyclictest233805-21sh11:14:211
134638993326,4cyclictest140457-21munin-plugin-st07:50:010
134638993324,6cyclictest421-21avahi-daemon11:28:330
134638993323,7cyclictest207871-21kernelversion10:34:180
134638993312,4cyclictest14-21ksoftirqd/010:04:290
13463899330,10cyclictest14-21ksoftirqd/009:14:180
134640993224,5cyclictest257598-21fschecks_time11:49:142
13464099322,24cyclictest153213-21ssh09:14:172
13464099321,5cyclictest171rcu_preempt10:49:222
13463999323,20cyclictest204739-21ssh10:29:231
134639993225,4cyclictest144509-21cron08:20:011
134639993225,3cyclictest0-21swapper/112:14:211
134639993225,3cyclictest0-21swapper/111:54:271
134639993224,5cyclictest138422-21grep07:35:001
134639993223,7cyclictest139527-21needreboot07:44:151
134638993228,2cyclictest0-21swapper/011:24:130
134638993228,2cyclictest0-21swapper/009:34:210
134638993226,4cyclictest228960-21sh11:05:010
134638993224,5cyclictest3499535-21cups-browsed11:52:470
134638993221,3cyclictest171rcu_preempt08:44:270
134638993220,9cyclictest218603-21mailstats10:49:300
134638993216,12cyclictest156785-21ssh09:19:240
134640993125,3cyclictest0-21swapper/212:05:012
134640993122,6cyclictest134649-21systemd-run07:10:012
134640993121,6cyclictest3499535-21cups-browsed07:46:072
13464099311,21cyclictest201450-21ssh10:24:252
134639993124,4cyclictest191644-21unixbench_singl10:09:321
134639993124,4cyclictest149368-21fschecks_count08:59:111
134639993123,4cyclictest0-21swapper/107:40:011
134639993122,6cyclictest181465-21kworker/u8:3-rpciod10:40:021
134639993122,6cyclictest181465-21kworker/u8:3-rpciod10:40:021
13463999312,23cyclictest166666-21ssh09:34:201
134639993121,7cyclictest422-21dbus-daemon07:55:011
134639993118,6cyclictest179907-21ssh09:54:041
13463899319,12cyclictest14-21ksoftirqd/007:29:190
134638993127,2cyclictest0-21swapper/010:24:080
134638993124,4cyclictest138671-21cron07:40:010
134638993121,7cyclictest957-21mta-sts-daemon07:44:290
134638993114,9cyclictest171rcu_preempt10:56:580
134640993022,5cyclictest144797-21munin-plugin-st08:25:012
134639993023,4cyclictest142494-21idleruntime-cro08:05:011
134639993021,6cyclictest3499535-21cups-browsed08:53:511
134639993016,11cyclictest197440-21idleruntime10:19:121
134638993025,2cyclictest0-21swapper/009:10:020
134638993021,6cyclictest191rcuc/012:13:460
134638993021,3cyclictest171rcu_preempt08:45:040
134638993015,3cyclictest171rcu_preempt08:59:170
13463899300,27cyclictest214779-21ssh10:44:170
134640992924,2cyclictest0-21swapper/207:35:012
134640992923,3cyclictest0-21swapper/212:40:012
13464099292,21cyclictest211609-21ssh10:39:262
13464099292,21cyclictest211609-21ssh10:39:262
134639992923,4cyclictest955-21mta-sts-daemon09:10:021
134639992923,3cyclictest0-21swapper/108:50:021
134639992920,6cyclictest148346-21unin-run08:45:021
134639992918,4cyclictest146533-21sshd08:34:261
134639992918,4cyclictest146533-21sshd08:34:261
13463899299,5cyclictest14-21ksoftirqd/010:55:020
13463899298,17cyclictest191rcuc/008:14:230
134640992822,3cyclictest0-21swapper/209:35:012
134640992821,4cyclictest0-21swapper/207:40:012
134640992820,5cyclictest3499535-21cups-browsed11:00:192
134640992820,5cyclictest163226-21if_wlan009:29:152
134640992820,5cyclictest0-21swapper/210:35:012
134639992813,3cyclictest171rcu_preempt08:10:021
13463899289,4cyclictest14-21ksoftirqd/007:59:210
134638992822,4cyclictest3499535-21cups-browsed08:00:470
134638992822,3cyclictest0-21swapper/007:55:010
134638992821,5cyclictest144551-21systemd-run08:20:010
134638992821,4cyclictest138646-21kworker/u8:8-xprtiod08:35:260
134639992721,4cyclictest955-21mta-sts-daemon07:27:261
134639992720,4cyclictest136667-21munin-plugin-st07:25:011
134639992720,4cyclictest136667-21munin-plugin-st07:25:011
134639992714,3cyclictest171rcu_preempt11:00:521
134638992723,2cyclictest0-21swapper/008:35:010
134638992723,2cyclictest0-21swapper/008:35:010
134638992713,3cyclictest171rcu_preempt07:21:340
134638992713,3cyclictest171rcu_preempt07:21:340
134638992710,3cyclictest171rcu_preempt07:16:170
134640992622,2cyclictest0-21swapper/211:09:322
134640992621,2cyclictest0-21swapper/208:55:012
134640992620,3cyclictest0-21swapper/208:05:012
134640992618,5cyclictest0-21swapper/209:00:012
13464099261,5cyclictest141672-21ntpq07:59:202
13463999265,4cyclictest28-21ksoftirqd/107:10:011
134639992619,4cyclictest140459-21munin-plugin-st07:50:011
134639992618,5cyclictest144986-21latency_hist08:25:021
134638992619,4cyclictest134737-21latency_hist07:10:020
134638992615,5cyclictest171rcu_preempt07:30:230
13464099251,6cyclictest136108-21cron07:15:012
13464099251,6cyclictest136108-21cron07:15:012
134640992516,6cyclictest150867-21cron09:10:012
134639992513,3cyclictest171rcu_preempt09:05:011
13463899259,3cyclictest14-21ksoftirqd/008:29:180
13463899253,7cyclictest151ktimers/009:05:030
13464099242,3cyclictest143696-21ntpq08:14:232
134640992418,2cyclictest0-21swapper/209:05:012
134640992416,3cyclictest351ktimers/207:44:232
134640992415,6cyclictest3499535-21cups-browsed08:07:012
134640992316,5cyclictest0-21swapper/208:40:002
13463899237,13cyclictest266-21systemd-journal08:25:010
13463899230,6cyclictest14-21ksoftirqd/008:10:020
134638992215,5cyclictest151ktimers/008:52:130
13464199212,10cyclictest0-21swapper/312:29:213
134640992116,2cyclictest0-21swapper/208:50:032
134640992116,2cyclictest0-21swapper/207:55:022
13464199204,9cyclictest0-21swapper/311:44:203
13464199203,9cyclictest0-21swapper/309:44:193
13464199203,9cyclictest0-21swapper/309:44:193
134640992014,3cyclictest0-21swapper/208:35:002
134640992014,3cyclictest0-21swapper/208:35:002
13464199193,7cyclictest0-21swapper/310:14:183
13464199193,7cyclictest0-21swapper/309:34:203
13464199193,11cyclictest0-21swapper/309:24:203
13464199192,9cyclictest0-21swapper/312:24:223
13464199192,11cyclictest0-21swapper/310:39:253
13464199192,11cyclictest0-21swapper/310:39:253
13464199192,10cyclictest0-21swapper/312:09:243
13464199192,10cyclictest0-21swapper/309:39:193
134640991913,3cyclictest0-21swapper/207:25:002
134640991913,3cyclictest0-21swapper/207:25:002
13464199184,9cyclictest0-21swapper/311:34:213
13464199183,9cyclictest0-21swapper/310:59:473
13464199183,8cyclictest0-21swapper/312:19:233
13464199183,10cyclictest0-21swapper/311:39:203
13464199182,9cyclictest0-21swapper/311:24:273
13464199182,11cyclictest0-21swapper/310:54:203
13464199181,9cyclictest0-21swapper/310:19:203
13464199174,8cyclictest0-21swapper/310:04:263
13464199174,8cyclictest0-21swapper/309:15:013
13464199173,9cyclictest0-21swapper/312:39:243
13464199173,9cyclictest0-21swapper/311:49:223
13464199173,9cyclictest0-21swapper/311:09:213
13464199173,7cyclictest0-21swapper/309:54:083
13464199173,10cyclictest0-21swapper/309:29:243
13464199172,8cyclictest0-21swapper/310:49:203
13464199172,10cyclictest0-21swapper/311:19:153
13464199165,8cyclictest0-21swapper/310:44:163
13464199163,9cyclictest0-21swapper/311:59:183
13464199163,8cyclictest0-21swapper/312:34:213
13464199163,8cyclictest0-21swapper/310:09:233
13464199163,8cyclictest0-21swapper/309:19:223
13464199163,7cyclictest0-21swapper/309:49:203
13464199162,9cyclictest0-21swapper/311:29:093
13464199162,9cyclictest0-21swapper/308:44:263
13464199162,8cyclictest0-21swapper/312:14:233
13464199162,8cyclictest0-21swapper/311:54:093
13464199162,8cyclictest0-21swapper/311:14:173
13464199162,8cyclictest0-21swapper/308:14:203
13464199162,10cyclictest0-21swapper/310:29:253
13464199153,7cyclictest0-21swapper/311:04:213
13464199152,9cyclictest0-21swapper/310:24:143
13464199152,8cyclictest0-21swapper/307:29:183
13464199152,10cyclictest0-21swapper/310:34:233
13464199151,8cyclictest0-21swapper/309:56:283
13464199144,5cyclictest0-21swapper/308:20:013
13464199143,6cyclictest0-21swapper/308:40:003
13464199142,8cyclictest0-21swapper/312:04:213
13464199142,8cyclictest0-21swapper/308:29:223
13464199142,7cyclictest0-21swapper/307:35:003
13464199142,6cyclictest0-21swapper/309:10:013
13464199142,6cyclictest0-21swapper/307:59:163
13464199142,6cyclictest0-21swapper/307:50:023
13464199141,8cyclictest0-21swapper/308:59:173
13464199133,6cyclictest0-21swapper/308:50:013
13464199132,7cyclictest0-21swapper/307:55:013
13464199132,7cyclictest0-21swapper/307:44:163
13464199131,7cyclictest0-21swapper/308:10:013
13464199131,7cyclictest0-21swapper/307:14:173
13464199131,7cyclictest0-21swapper/307:14:173
13464199127,3cyclictest0-21swapper/307:10:013
13464199122,6cyclictest0-21swapper/307:20:013
13464199112,6cyclictest0-21swapper/308:25:013
13464199111,6cyclictest0-21swapper/308:05:033
13464199104,3cyclictest0-21swapper/307:36:083
13464199101,6cyclictest0-21swapper/307:25:013
13464199101,6cyclictest0-21swapper/307:25:013
1346419991,5cyclictest0-21swapper/308:55:013
1346419991,5cyclictest0-21swapper/308:35:013
1346419991,5cyclictest0-21swapper/308:35:013
1346419984,3cyclictest0-21swapper/309:02:413
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional