You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-18 - 04:08

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Phytec
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot5s.osadl.org (updated Tue Nov 18, 2025 00:44:48)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
935393995238,5cyclictest934988-21kworker/u8:1+events_unbound19:33:063
935380995213,29cyclictest0-21swapper/023:00:250
935380995213,29cyclictest0-21swapper/023:00:250
93538099520,43cyclictest0-21swapper/020:55:170
93538099520,30cyclictest956825-21apt-get20:10:130
935384995137,10cyclictest231ktimers/121:10:171
935384995137,10cyclictest231ktimers/121:10:171
935393995035,11cyclictest391ktimers/323:45:133
935384994927,8cyclictest131rcu_preempt20:00:011
935389994740,4cyclictest1012046-21idleruntime-cro22:50:012
935384994736,7cyclictest966814-21cron20:40:001
93538099473,26cyclictest943180-21apt-get19:30:120
935380994725,17cyclictest0-21swapper/020:00:170
935384994638,4cyclictest998395-21idleruntime-cro22:10:011
935393994538,4cyclictest939421-21latency_hist19:20:023
935384994535,6cyclictest1015449-21idleruntime-cro23:00:001
93538099453,20cyclictest0-21swapper/000:17:460
93538099453,20cyclictest0-21swapper/000:17:460
93538099451,38cyclictest1000328-21apt-get22:15:160
935380994511,30cyclictest1013269-21grep22:50:250
93538099450,38cyclictest0-21swapper/020:45:130
935393994436,5cyclictest959883-21kworker/u8:2+events_unbound20:23:203
935389994433,7cyclictest963837-21fschecks_time20:30:162
935380994417,24cyclictest1038081-21fschecks_time00:05:160
93538099441,37cyclictest1002484-21/usr/sbin/munin22:20:180
935389994336,5cyclictest977901-21kworker/u8:3+events_unbound21:18:312
935389994332,8cyclictest994546-21cron22:00:012
935389994332,7cyclictest0-21swapper/222:20:162
93538099431,39cyclictest995640-21/usr/sbin/munin22:00:230
93538099431,37cyclictest960359-21/usr/sbin/munin20:20:140
93538099431,37cyclictest941478-21munin-node19:25:120
93538099431,37cyclictest1046714-21/usr/sbin/munin00:30:140
93538099431,35cyclictest965357-21apt-get20:35:180
93538099431,24cyclictest1019213-21diskstats23:10:150
935393994235,5cyclictest982133-21kworker/u8:2+events_unbound21:33:363
935384994235,4cyclictest984115-21grep21:30:011
935380994235,4cyclictest998394-21latency_hist22:10:010
935380994231,7cyclictest992382-21grep21:50:270
93538099421,38cyclictest979327-21/usr/sbin/munin21:15:180
93538099421,37cyclictest971530-21grep20:50:270
93538099421,35cyclictest1017506-21/usr/sbin/munin23:05:170
93538099421,35cyclictest1017506-21/usr/sbin/munin23:05:170
935384994134,4cyclictest1029098-21munin-run23:40:011
935380994130,7cyclictest1000152-21seq22:15:010
93538099412,35cyclictest1049358-21unixbench_singl00:35:290
93538099411,37cyclictest1014993-21/usr/sbin/munin22:55:270
93538099411,35cyclictest951710-21/usr/sbin/munin19:55:170
93538099411,34cyclictest1025919-21apt-get23:30:120
935389994033,4cyclictest1039342-21cut00:10:002
935389994027,9cyclictest311ktimers/223:45:122
935384994031,5cyclictest949776-21idleruntime-cro19:50:001
935380994021,15cyclictest0-21swapper/022:00:000
935380994019,18cyclictest13950irq/36-mmc122:35:010
93538099401,36cyclictest981172-21/usr/sbin/munin21:20:210
93538099401,36cyclictest976265-21/usr/sbin/munin21:05:240
93538099401,36cyclictest1034714-21/usr/sbin/munin23:55:180
93538099401,35cyclictest965107-21cron20:35:010
93538099401,35cyclictest950542-21/usr/sbin/munin19:50:220
93538099401,35cyclictest950542-21/usr/sbin/munin19:50:220
93538099401,34cyclictest935939-21/usr/sbin/munin19:10:210
93538099400,37cyclictest0-21swapper/019:35:140
93539399390,16cyclictest0-21swapper/300:00:133
935389993931,4cyclictest0-21swapper/222:45:012
935384993932,4cyclictest971941-21cron20:55:011
93538099392,32cyclictest946603-21apt-get19:40:160
93538099392,32cyclictest1027739-21diskstats23:35:150
93538099392,31cyclictest939767-21apt-get19:20:060
935380993921,14cyclictest0-21swapper/021:30:160
93538099391,35cyclictest988344-21/usr/sbin/munin21:40:190
93538099391,35cyclictest968194-21cron20:41:000
93538099391,35cyclictest1039778-21apt-get00:10:120
93538099391,35cyclictest1023542-21/usr/sbin/munin23:20:270
93538099391,34cyclictest977642-21irqstats21:10:220
93538099391,34cyclictest977642-21irqstats21:10:210
93538099391,34cyclictest1043253-21awk00:20:130
93538099391,34cyclictest1024741-21/usr/sbin/munin23:25:230
93538099391,33cyclictest1007149-21apt-get22:35:150
93538099391,18cyclictest0-21swapper/022:30:010
93538099390,34cyclictest1020799-21apt-get23:15:150
935393993831,4cyclictest977076-21idleruntime-cro21:10:003
935389993827,6cyclictest0-21swapper/222:05:272
935384993827,7cyclictest0-21swapper/119:20:011
935380993827,7cyclictest987258-21grep21:35:280
93538099382,33cyclictest962293-21/usr/sbin/munin20:25:210
93538099382,33cyclictest938508-21/usr/sbin/munin19:15:240
93538099381,34cyclictest990239-21ntp_states21:45:210
93538099381,34cyclictest982804-21/usr/sbin/munin21:25:190
93538099381,34cyclictest1030420-21users23:40:300
93538099381,34cyclictest1011022-21/usr/sbin/munin22:45:240
93538099381,33cyclictest955448-21/usr/sbin/munin20:05:200
93538099381,32cyclictest948299-21munin-node19:45:150
93538099381,32cyclictest1032747-21apt-get23:50:170
93538099380,32cyclictest0-21swapper/021:00:140
935393993729,5cyclictest1013451-21kworker/u8:2+events_unbound23:10:423
93538099371,34cyclictest1044905-21apt-get00:25:130
93538099371,33cyclictest1037371-21cron00:01:590
93538099371,33cyclictest1009210-21irqstats22:40:220
93538099371,31cyclictest958660-21awk20:15:160
935393993626,6cyclictest0-21swapper/300:09:593
935393993625,8cyclictest982225-21latency_hist21:25:013
935393993619,6cyclictest1005442-21kworker/u8:1+events_unbound22:51:383
93538099361,32cyclictest1031158-21diskstats23:45:160
935393993527,4cyclictest0-21swapper/300:25:003
935393993524,8cyclictest1001794-21auditctl22:20:013
935393993523,8cyclictest941242-21sh19:25:013
935384993527,4cyclictest0-21swapper/122:05:011
935384993527,4cyclictest0-21swapper/100:20:151
935384993526,6cyclictest1047623-21unixbench_multi00:30:261
935384993521,8cyclictest131rcu_preempt21:08:131
935393993427,5cyclictest971835-21kworker/u8:0+events_unbound21:03:323
935393993423,8cyclictest1015463-21sendmail23:00:003
935393993422,8cyclictest952769-21grep19:55:283
935389993428,4cyclictest1023911-21kworker/u8:1+events_unbound23:33:592
935389993427,4cyclictest0-21swapper/219:15:002
935384993424,6cyclictest0-21swapper/120:40:261
935384993423,6cyclictest0-21swapper/119:20:181
935393993318,6cyclictest936762-21kworker/u8:0+events_unbound19:12:283
93538999335,5cyclictest131rcu_preempt21:30:132
935389993325,5cyclictest1030425-21kworker/u8:0+events_unbound23:51:422
935389993325,4cyclictest0-21swapper/219:25:172
935389993322,7cyclictest0-21swapper/223:25:292
935384993325,4cyclictest0-21swapper/123:20:271
935384993324,5cyclictest0-21swapper/100:05:001
935384993322,6cyclictest0-21swapper/122:40:281
935384993321,9cyclictest221rcuc/122:39:041
935384993321,7cyclictest0-21swapper/120:25:281
935384993313,5cyclictest131rcu_preempt22:20:201
935393993210,8cyclictest0-21swapper/323:30:253
935389993225,4cyclictest0-21swapper/219:20:012
93538999322,4cyclictest131rcu_preempt00:35:152
935389993222,6cyclictest0-21swapper/222:30:262
935389993219,5cyclictest948874-21kworker/u8:2+events_unbound19:49:182
93538499325,6cyclictest24-21ksoftirqd/120:35:011
935384993222,7cyclictest221rcuc/100:14:141
935393993125,4cyclictest1034015-21kworker/u8:2+events_unbound23:59:043
935393993124,4cyclictest1049795-21sendmail00:40:013
935393993123,4cyclictest966814-21cron20:40:013
935393993112,5cyclictest131rcu_preempt23:40:003
93539399310,6cyclictest131rcu_preempt23:15:133
93538999318,6cyclictest967412-21kworker/u8:1+events_unbound20:47:302
93538999314,4cyclictest131rcu_preempt23:35:132
935389993124,3cyclictest0-21swapper/221:05:272
935389993122,6cyclictest0-21swapper/219:25:012
935389993121,8cyclictest1000089-21auditctl22:15:012
935389993121,7cyclictest0-21swapper/222:35:272
935389993121,6cyclictest0-21swapper/221:20:272
935389993121,6cyclictest0-21swapper/220:40:272
935389993121,6cyclictest0-21swapper/220:00:252
93538499317,5cyclictest24-21ksoftirqd/123:50:191
93538499315,5cyclictest24-21ksoftirqd/100:00:001
935384993123,4cyclictest0-21swapper/121:55:281
935384993122,6cyclictest1001321-21grep22:15:301
935384993120,8cyclictest221rcuc/123:12:121
93538099311,24cyclictest935483-21latency_hist19:10:000
935393993023,5cyclictest1011731-21kworker/u8:0+events_unbound23:00:433
935393993023,5cyclictest1011731-21kworker/u8:0+events_unbound23:00:433
935393993023,4cyclictest0-21swapper/322:45:303
935393993021,6cyclictest0-21swapper/320:20:003
935393993018,3cyclictest131rcu_preempt22:10:023
93539399301,14cyclictest965111-21cron20:35:013
935389993024,4cyclictest989093-21kworker/u8:1+events_unbound22:18:462
935389993020,6cyclictest0-21swapper/219:30:272
93538499308,7cyclictest24-21ksoftirqd/121:30:021
93538499304,7cyclictest24-21ksoftirqd/100:05:151
935384993021,6cyclictest999634-21unixbench_multi22:10:251
935384993021,6cyclictest221rcuc/122:30:461
935384993020,7cyclictest979217-21cut21:15:161
935384993020,5cyclictest0-21swapper/100:35:281
935384993019,7cyclictest973178-21grep20:55:301
935384993019,6cyclictest131rcu_preempt21:52:191
935384993018,3cyclictest131rcu_preempt23:40:191
935384993016,7cyclictest948874-21kworker/u8:2+events_unbound19:50:581
935384993016,7cyclictest948874-21kworker/u8:2+events_unbound19:50:571
935384993015,9cyclictest131rcu_preempt20:09:391
935384993011,5cyclictest131rcu_preempt19:43:291
935393992922,4cyclictest0-21swapper/321:25:293
935393992922,3cyclictest0-21swapper/322:35:013
935393992921,5cyclictest987295-21unixbench_singl21:35:273
93539399291,7cyclictest0-21swapper/300:10:113
935393992912,4cyclictest994255-21kworker/u8:0+events_freezable_power_22:04:153
935389992922,5cyclictest942844-21kworker/u8:0+events_unbound20:10:022
935389992922,4cyclictest978778-21idleruntime-cro21:15:012
935389992922,4cyclictest978778-21idleruntime-cro21:15:002
935389992922,4cyclictest0-21swapper/222:55:172
935389992922,4cyclictest0-21swapper/221:30:012
935389992920,6cyclictest0-21swapper/223:15:262
935389992918,7cyclictest0-21swapper/200:24:052
93538499296,6cyclictest24-21ksoftirqd/120:15:001
93538499295,8cyclictest24-21ksoftirqd/120:45:271
93538499295,6cyclictest24-21ksoftirqd/123:05:191
93538499295,6cyclictest24-21ksoftirqd/123:05:181
935384992923,4cyclictest221rcuc/123:33:001
935384992921,4cyclictest0-21swapper/121:00:271
935384992921,4cyclictest0-21swapper/119:10:191
935384992920,5cyclictest0-21swapper/120:05:001
93538499291,9cyclictest24-21ksoftirqd/119:30:201
935384992916,4cyclictest131rcu_preempt19:28:121
935384992914,9cyclictest131rcu_preempt23:04:171
935384992914,9cyclictest131rcu_preempt23:04:171
93539399288,11cyclictest40-21ksoftirqd/321:00:013
935393992822,4cyclictest992851-21latency_hist21:55:013
935393992822,3cyclictest0-21swapper/323:20:243
935393992819,7cyclictest954665-21kworker/u8:1+events_unbound20:05:163
935393992817,5cyclictest131rcu_preempt22:11:223
93539399281,15cyclictest0-21swapper/300:30:113
93539399281,14cyclictest969382-21/usr/sbin/munin20:45:243
93539399280,15cyclictest0-21swapper/323:40:163
935389992819,5cyclictest0-21swapper/220:35:312
935389992817,7cyclictest0-21swapper/221:50:282
93538999281,2cyclictest0-21swapper/220:10:142
93538499288,3cyclictest231ktimers/121:40:151
93538499287,7cyclictest24-21ksoftirqd/100:15:261
93538499287,7cyclictest24-21ksoftirqd/100:15:251
93538499286,6cyclictest24-21ksoftirqd/100:30:011
93538499283,4cyclictest0-21swapper/121:45:211
935384992818,7cyclictest231ktimers/123:45:181
935384992814,10cyclictest1013265-21grep22:50:261
93539399278,9cyclictest131rcu_preempt21:45:183
935393992721,4cyclictest1023911-21kworker/u8:1+events_unbound23:29:433
935393992720,4cyclictest0-21swapper/319:45:303
935393992720,3cyclictest0-21swapper/320:25:193
935393992714,5cyclictest1030425-21kworker/u8:0+events_unbound23:51:193
935389992720,5cyclictest1044653-21kworker/u8:2+events_unbound00:29:112
935389992719,4cyclictest0-21swapper/220:55:292
935389992717,6cyclictest0-21swapper/223:55:262
935389992717,6cyclictest0-21swapper/223:05:202
935389992717,6cyclictest0-21swapper/223:05:202
935389992717,6cyclictest0-21swapper/223:00:272
935389992717,6cyclictest0-21swapper/223:00:262
93538499277,17cyclictest231ktimers/120:19:531
93538499277,12cyclictest0-21swapper/121:21:411
93538499274,6cyclictest24-21ksoftirqd/122:45:241
935384992720,4cyclictest946367-21sendmail19:40:011
935384992718,6cyclictest231ktimers/121:35:231
935384992717,7cyclictest231ktimers/123:25:221
935384992717,7cyclictest231ktimers/122:25:251
93539399268,5cyclictest131rcu_preempt21:40:263
93539399267,11cyclictest40-21ksoftirqd/319:40:013
93539399267,11cyclictest131rcu_preempt20:05:013
93539399263,11cyclictest342341-21kworker/3:2-events00:17:193
93539399263,11cyclictest342341-21kworker/3:2-events00:17:193
935393992620,4cyclictest1003447-21kworker/u8:0+events_unbound22:28:463
935393992619,5cyclictest989093-21kworker/u8:1+events_unbound21:55:233
935393992617,6cyclictest1045916-21unixbench_multi00:25:263
93539399261,12cyclictest0-21swapper/319:44:383
93539399260,7cyclictest40-21ksoftirqd/323:05:193
93539399260,7cyclictest40-21ksoftirqd/323:05:183
93538999269,5cyclictest951178-21kworker/u8:1+events_unbound19:56:532
935389992620,4cyclictest942844-21kworker/u8:0+events_unbound20:20:252
935389992620,4cyclictest934988-21kworker/u8:1+events_unbound19:45:012
935389992617,6cyclictest0-21swapper/221:35:272
935389992617,6cyclictest0-21swapper/200:10:142
935389992617,6cyclictest0-21swapper/200:00:272
935389992617,4cyclictest0-21swapper/222:03:412
935384992618,6cyclictest231ktimers/123:19:411
93539399258,10cyclictest131rcu_preempt19:25:163
93539399256,8cyclictest40-21ksoftirqd/322:40:293
935393992519,3cyclictest0-21swapper/322:20:253
935393992518,5cyclictest942844-21kworker/u8:0+events_unbound20:11:383
935393992517,6cyclictest967615-21ntp_kernel_pll_20:40:233
935393992516,6cyclictest0-21swapper/322:35:183
935393992513,9cyclictest40-21ksoftirqd/321:10:253
935393992513,9cyclictest40-21ksoftirqd/321:10:243
935389992519,3cyclictest0-21swapper/222:25:172
935389992518,5cyclictest1020477-21kworker/u8:0+events_unbound23:14:402
935389992518,4cyclictest0-21swapper/219:35:172
935389992516,5cyclictest0-21swapper/220:50:262
935389992514,8cyclictest0-21swapper/219:50:272
935389992514,8cyclictest0-21swapper/219:50:272
93539399249,5cyclictest131rcu_preempt19:50:283
93539399249,5cyclictest131rcu_preempt19:50:273
935389992418,3cyclictest0-21swapper/221:00:182
935389992418,3cyclictest0-21swapper/220:15:172
935389992413,3cyclictest32-21ksoftirqd/221:40:252
93539399238,8cyclictest40-21ksoftirqd/321:20:013
93539399237,9cyclictest40-21ksoftirqd/320:50:203
935389992316,4cyclictest0-21swapper/200:15:262
935389992316,4cyclictest0-21swapper/200:15:252
935389992313,7cyclictest1047619-21unixbench_multi00:30:262
935389992312,8cyclictest1022621-21ls23:20:142
935389992311,9cyclictest962400-21memory20:25:222
93538999231,11cyclictest0-21swapper/222:50:222
935389992216,3cyclictest0-21swapper/221:45:162
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional