You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-20 - 12:00

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack7slot3s.osadl.org (updated Thu Nov 20, 2025 00:43:38)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1685421700,3sleep2811499cyclictest20:00:222
81129910036,32cyclictest0-21swapper/023:15:190
128142950,6sleep3811599cyclictest21:14:103
254452930,2sleep10-21swapper/121:30:221
254452930,2sleep10-21swapper/121:30:221
811299878,67cyclictest0-21swapper/023:30:190
811299878,67cyclictest0-21swapper/023:30:190
811299858,63cyclictest10507-21unixbench_multi22:35:330
811299843,63cyclictest0-21swapper/019:10:070
8112998052,7cyclictest7750irq/56-dwc_otg21:25:190
8112998021,37cyclictest0-21swapper/020:35:190
811599797,27cyclictest0-21swapper/320:35:193
811299796,63cyclictest30301-21ssh23:45:230
156162790,2sleep00-21swapper/022:00:240
8112997544,9cyclictest7750irq/56-dwc_otg22:30:190
8112997527,34cyclictest0-21swapper/020:25:100
811299752,60cyclictest18252-21idleruntime-cro20:05:040
811299746,56cyclictest8432-21missed_timers21:05:260
98492730,3sleep29848-21fschecks_time00:00:232
811299732,57cyclictest0-21swapper/020:10:340
811299727,49cyclictest17007-21/usr/sbin/munin22:45:200
811299726,51cyclictest0-21swapper/021:30:150
811299726,51cyclictest0-21swapper/021:30:150
811299724,65cyclictest11604-21aten_rbpower_cu21:55:190
811299719,53cyclictest4769-21grep23:10:330
811299704,54cyclictest18385-21cron23:30:050
811299702,49cyclictest0-21swapper/023:55:060
811299702,49cyclictest0-21swapper/023:55:060
8114996915,19cyclictest7950irq/56-dwc_otg_00:05:192
811299691,53cyclictest6704-21ls23:55:330
190402690,2sleep10-21swapper/100:11:491
811299685,53cyclictest30742-21fschecks_time22:20:220
294142680,3sleep129384-21switchtime23:00:321
294142680,3sleep129384-21switchtime23:00:321
811399677,20cyclictest7750irq/56-dwc_otg00:05:191
811299658,54cyclictest24409-21ssh00:20:110
811299658,54cyclictest24409-21ssh00:20:110
8112996551,7cyclictest0-21swapper/000:28:530
8112996459,3cyclictest0-21swapper/000:39:580
811299644,50cyclictest6110-21basename21:00:200
811299641,42cyclictest23136-21cut22:10:200
54652640,2sleep20-21swapper/222:30:092
811299637,43cyclictest0-21swapper/023:41:370
811299632,44cyclictest0-21swapper/022:55:250
811299631,47cyclictest1943-21cpuspeed22:25:190
246422630,1sleep20-21swapper/222:11:302
811299628,44cyclictest0-21swapper/021:24:480
811299623,50cyclictest7441-21latency_hist21:50:050
811299614,45cyclictest2285-21ssh23:07:590
811299614,39cyclictest0-21swapper/021:35:560
811299612,48cyclictest3226-21chrt21:44:140
8112996032,6cyclictest26218-21latency_hist23:40:050
8112996011,36cyclictest0-21swapper/021:11:140
811299598,34cyclictest0-21swapper/000:12:140
811299595,42cyclictest0-21swapper/022:17:540
811299593,45cyclictest22164-21taskset22:09:150
811299593,39cyclictest0-21swapper/019:46:350
811299593,39cyclictest0-21swapper/019:46:350
811299588,39cyclictest0-21swapper/023:23:150
811299588,34cyclictest12209-21ssh00:03:330
811299586,38cyclictest0-21swapper/021:16:470
8112995838,5cyclictest2159-21modprobe20:50:050
811599577,18cyclictest0-21swapper/323:55:113
8112995737,6cyclictest7750irq/56-dwc_otg20:20:260
811299573,45cyclictest820-21ssh00:31:180
811299573,39cyclictest30495-21ssh23:02:090
811299573,39cyclictest30495-21ssh23:02:090
8115995615,23cyclictest19484-21ntp_offset23:30:283
8115995615,23cyclictest19484-21ntp_offset23:30:283
811299567,40cyclictest9427-21diskmemload22:53:470
811299559,33cyclictest15317-21ssh22:42:350
811299553,49cyclictest10443-21latency_hist19:30:050
811299553,41cyclictest28959-21taskset20:32:300
8112995523,14cyclictest7750irq/56-dwc_otg20:15:290
811299551,39cyclictest16297-21ssh00:09:070
811299551,38cyclictest13515-21idleruntime19:51:300
811599534,17cyclictest7950irq/56-dwc_otg_20:30:223
811299538,36cyclictest0-21swapper/019:16:430
811299538,31cyclictest0-21swapper/000:17:210
811299536,38cyclictest9749-21modprobe19:25:050
811299536,36cyclictest0-21swapper/019:59:540
811299536,34cyclictest9973-21ssh21:52:370
8115995218,12cyclictest23669-21ssh22:54:043
8115995217,13cyclictest15286-21ssh22:42:303
811299529,35cyclictest32075-21ls20:40:320
811299528,34cyclictest0-21swapper/019:32:470
811299527,32cyclictest0-21swapper/019:43:280
811599516,13cyclictest7850irq/56-dwc_otg_00:25:183
811599516,10cyclictest7850irq/56-dwc_otg_00:20:173
811599516,10cyclictest7850irq/56-dwc_otg_00:20:173
8115995121,16cyclictest32273-21/usr/sbin/munin23:05:153
8112995115,33cyclictest101ktimersoftd/020:05:270
8115995018,14cyclictest5773-21idleruntime-cro21:00:053
8115995017,12cyclictest7598-21sort21:50:063
8115995013,13cyclictest21232-21rm21:25:113
8115995011,13cyclictest25554-21uname21:30:243
8115995011,13cyclictest25554-21uname21:30:243
8115995010,14cyclictest2078-21ssh00:34:053
811299502,37cyclictest0-21swapper/019:38:090
8115994919,18cyclictest16482-21sshd00:09:293
8115994910,13cyclictest24067-21idleruntime-cro00:20:043
197392490,1sleep20-21swapper/222:05:262
811299482,31cyclictest3543-21grep20:50:360
811599478,18cyclictest3415-21chrt22:26:203
811599477,11cyclictest11578-21/usr/sbin/munin21:55:193
8115994717,12cyclictest19360-21sh22:48:063
8115994716,12cyclictest14443-21rm23:24:463
811599471,5cyclictest7950irq/56-dwc_otg_20:25:163
8115994712,18cyclictest0-21swapper/320:05:293
8115994712,14cyclictest31018-21ssh22:20:253
8114994726,6cyclictest7850irq/56-dwc_otg_20:15:182
8112994720,17cyclictest0-21swapper/020:55:530
811599468,25cyclictest0-21swapper/321:35:183
8115994620,11cyclictest11870-21sshd19:40:493
8115994617,11cyclictest20829-21ssh22:06:263
8115994612,14cyclictest8580-21/usr/sbin/munin21:05:293
811499468,11cyclictest0-21swapper/220:30:192
811499465,7cyclictest13077-21ssh22:40:202
8113994618,11cyclictest2650850wget23:40:161
76142463,35sleep00-21swapper/019:07:030
76142463,35sleep00-21swapper/019:07:030
811599457,26cyclictest0-21swapper/320:00:213
811599457,26cyclictest0-21swapper/300:10:193
811599457,24cyclictest0-21swapper/322:35:193
8115994515,11cyclictest9762-21latency_hist19:25:063
8115994514,12cyclictest10216-21ssh23:18:513
8115994512,17cyclictest24897-21sh22:12:023
8115994510,12cyclictest5062-21rm23:10:553
8113994516,14cyclictest9427-21diskmemload21:27:321
811599446,24cyclictest17536-21ssh21:20:173
8115994414,17cyclictest7850irq/56-dwc_otg_23:05:003
8115994414,17cyclictest7850irq/56-dwc_otg_23:05:003
8115994412,12cyclictest1937-21ssh21:41:313
8114994423,5cyclictest1120-21runrttasks22:20:052
811599436,25cyclictest0-21swapper/320:40:203
8115994322,8cyclictest6009-21apt-get21:00:183
8115994316,12cyclictest7386-21ssh22:31:443
8115994314,11cyclictest26305-21ssh22:56:343
8115994314,11cyclictest1066-21ssh23:49:343
8115994310,11cyclictest10829-21taskset19:33:133
811399434,5cyclictest7950irq/56-dwc_otg_00:15:191
811399433,23cyclictest0-21swapper/119:10:071
8115994210,18cyclictest7750irq/56-dwc_otg00:01:093
811499429,8cyclictest7950irq/56-dwc_otg_22:55:182
8114994224,6cyclictest0-21swapper/220:45:052
8114994213,9cyclictest0-21swapper/223:45:392
811399429,7cyclictest7850irq/56-dwc_otg_22:55:181
811399422,5cyclictest25231-21ntp_states00:20:291
811399422,5cyclictest25231-21ntp_states00:20:291
8113994218,6cyclictest9814-21fschecks_time00:00:231
78392427,21sleep37934-21latency_hist19:10:053
78392427,21sleep37934-21latency_hist19:10:053
297282420,5sleep2811499cyclictest21:35:302
811599414,20cyclictest0-21swapper/320:19:123
8115994113,10cyclictest5493-21ssh00:38:453
8115994110,10cyclictest7750irq/56-dwc_otg21:52:393
811499417,16cyclictest7950irq/56-dwc_otg_19:10:072
8113994122,12cyclictest8562-21ntp_states21:05:291
811399411,11cyclictest7750irq/56-dwc_otg20:10:191
8113994110,6cyclictest30116-21/usr/sbin/munin21:35:341
811599408,11cyclictest27148-21ssh23:40:253
811599407,14cyclictest17245-21rm23:27:403
8115994017,18cyclictest0-21swapper/319:10:073
8115994013,12cyclictest16778-21taskset21:19:143
811399405,3cyclictest0-21swapper/100:25:191
811399404,7cyclictest18414-21sh23:30:051
811399402,6cyclictest0-21swapper/121:10:311
8113994010,14cyclictest23-21ksoftirqd/100:35:191
8115993914,11cyclictest16942-21ssh22:01:143
8115993913,12cyclictest23858-21ssh23:35:373
8114993915,11cyclictest28676-21cat00:25:272
811499391,3cyclictest0-21swapper/223:14:042
8114993913,12cyclictest18906-21sh22:05:052
811499391,28cyclictest10482-21ls19:30:052
811399391,8cyclictest0-21swapper/120:30:191
8115993816,10cyclictest1953-21diskstats23:50:253
8115993816,10cyclictest1953-21diskstats23:50:253
8115993810,11cyclictest27585-21ssh22:15:323
8114993817,5cyclictest30367-21timerandwakeup20:35:322
811499381,24cyclictest8094-21cut21:05:222
8113993816,12cyclictest18018-21sh21:20:231
8113993812,13cyclictest9427-21diskmemload22:12:011
8113993811,7cyclictest16681-21/usr/sbin/munin20:00:221
8113993811,5cyclictest7791-21latency_hist21:05:061
262982380,2sleep10-21swapper/120:25:251
145742380,2sleep10-21swapper/123:25:041
811599377,11cyclictest451-21rs:main3
8115993728,7cyclictest15341-21irqstats19:55:233
8115993720,13cyclictest2506-21aten_rbpower_vo20:50:223
811599371,24cyclictest0-21swapper/319:54:393
8115993711,9cyclictest0-21swapper/320:24:413
8114993717,5cyclictest647-21/usr/sbin/munin23:05:292
811499371,5cyclictest25592-21latency_hist20:25:062
8114993715,5cyclictest7850irq/56-dwc_otg_23:50:262
8114993715,5cyclictest7850irq/56-dwc_otg_23:50:262
8114993711,14cyclictest0-21swapper/200:10:352
811399376,10cyclictest7750irq/56-dwc_otg20:15:181
811399374,4cyclictest7950irq/56-dwc_otg_23:45:271
811399371,27cyclictest7549-21ls23:15:201
811399371,24cyclictest8077-21rm22:33:121
8114993625,4cyclictest30061-21ssh23:01:172
8114993625,4cyclictest30061-21ssh23:01:172
811499361,16cyclictest7850irq/56-dwc_otg_21:15:242
811399368,9cyclictest7950irq/56-dwc_otg_22:05:051
811399367,24cyclictest0-21swapper/120:05:181
811399366,5cyclictest7850irq/56-dwc_otg_22:20:051
8113993616,10cyclictest20691-21sh23:31:581
8113993616,10cyclictest20691-21sh23:31:581
8113993614,4cyclictest14366-21grep19:53:221
8113993611,8cyclictest14015-21tune2fs21:15:211
811399361,13cyclictest0-21swapper/119:27:281
811599356,22cyclictest0-21swapper/319:48:563
811599356,22cyclictest0-21swapper/319:48:563
8115993526,7cyclictest20497-21df20:10:233
8114993519,8cyclictest21668-21sh23:34:002
8114993519,8cyclictest21668-21sh23:34:002
8114993515,5cyclictest182750wget22:25:182
8114993514,6cyclictest1120-21runrttasks21:25:322
811399356,7cyclictest0-21swapper/121:50:201
811399353,7cyclictest7850irq/56-dwc_otg_20:40:281
811399353,6cyclictest19562-21ssh22:05:231
811399353,4cyclictest0-21swapper/122:40:281
811399351,5cyclictest21264-21ls22:50:251
811399351,20cyclictest5397-21ssh21:45:521
811499348,13cyclictest0-21swapper/221:57:072
811499347,24cyclictest0-21swapper/220:05:182
811499342,13cyclictest7950irq/56-dwc_otg_23:20:172
811499341,19cyclictest7950irq/56-dwc_otg_21:55:052
8114993411,7cyclictest21265-21sh22:50:252
811399349,16cyclictest0-21swapper/123:56:001
811399347,4cyclictest7750irq/56-dwc_otg00:30:271
811399345,3cyclictest0-21swapper/122:45:331
811399344,11cyclictest25544-21apt-config20:25:051
811399342,16cyclictest0-21swapper/123:10:281
8113993412,7cyclictest825-21fschecks_count20:45:221
811399341,12cyclictest0-21swapper/123:38:591
811599338,6cyclictest0-21swapper/319:35:533
811499338,14cyclictest0-21swapper/200:30:152
811499334,4cyclictest0-21swapper/221:32:592
811499334,4cyclictest0-21swapper/221:32:592
811499333,17cyclictest7950irq/56-dwc_otg_23:55:132
811499332,7cyclictest0-21swapper/221:10:332
811499331,5cyclictest5371-21users20:55:312
811499331,17cyclictest7950irq/56-dwc_otg_21:45:592
811399336,9cyclictest0-21swapper/123:05:281
811399336,7cyclictest7850irq/56-dwc_otg_22:25:241
811399336,5cyclictest7750irq/56-dwc_otg22:00:041
811399332,5cyclictest1120-21runrttasks19:43:331
8113993315,4cyclictest221ktimersoftd/123:50:211
8113993315,4cyclictest221ktimersoftd/123:50:211
8113993312,9cyclictest0-21swapper/122:20:221
811399331,12cyclictest15183-21grep19:55:191
8113993310,5cyclictest0-21swapper/120:35:171
8113993310,15cyclictest4245-21basename20:55:171
811599327,13cyclictest0-21swapper/319:16:513
811499328,4cyclictest0-21swapper/221:40:452
811499328,16cyclictest140-21systemd-journal19:45:202
811499328,16cyclictest140-21systemd-journal19:45:202
811499322,16cyclictest15886-21grep23:25:342
8114993213,4cyclictest24616-21cpu00:20:202
8114993213,4cyclictest24616-21cpu00:20:202
811499321,19cyclictest17009-21ls22:45:202
811399326,5cyclictest9323-21ssh22:35:191
811399326,4cyclictest7850irq/56-dwc_otg_20:50:311
811399321,4cyclictest467-21fschecks_count21:40:211
811399321,10cyclictest7950irq/56-dwc_otg_19:20:051
71262327,15sleep10-21swapper/119:05:341
71262327,15sleep10-21swapper/119:05:341
289612321,2sleep20-21swapper/223:43:092
8115993119,10cyclictest2005-21sshd20:48:253
811499318,6cyclictest0-21swapper/222:35:192
811499315,5cyclictest7750irq/56-dwc_otg22:23:352
811499311,20cyclictest14177-21switchtime19:53:192
811499304,5cyclictest7850irq/56-dwc_otg_00:15:282
811499302,3cyclictest7850irq/56-dwc_otg_21:20:252
811499301,4cyclictest7850irq/56-dwc_otg_20:55:052
811499301,19cyclictest20528-21cut20:10:242
811499301,14cyclictest7950irq/56-dwc_otg_19:40:562
811399309,5cyclictest7950irq/56-dwc_otg_19:25:051
811499294,4cyclictest7850irq/56-dwc_otg_21:00:332
811499294,4cyclictest7750irq/56-dwc_otg23:35:252
811499294,11cyclictest7850irq/56-dwc_otg_23:16:542
811499292,5cyclictest3105-21df_inode00:35:222
811499291,7cyclictest0-21swapper/220:45:222
811499291,19cyclictest0-21swapper/219:30:232
811399294,5cyclictest0-21swapper/119:31:141
67012296,12sleep20-21swapper/219:05:272
67012296,12sleep20-21swapper/219:05:272
811499281,5cyclictest7850irq/56-dwc_otg_19:25:052
811399282,7cyclictest0-21swapper/119:49:581
811399282,7cyclictest0-21swapper/119:49:581
811399281,8cyclictest1120-21runrttasks19:39:481
811499272,4cyclictest7750irq/56-dwc_otg20:25:252
811499272,3cyclictest0-21swapper/219:39:492
811499272,11cyclictest7750irq/56-dwc_otg19:15:142
811499262,9cyclictest15182-21tune2fs19:55:192
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional