You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-16 - 10:38

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Phytec
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot5s.osadl.org (updated Sun Nov 16, 2025 00:44:48)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3983742995850,5cyclictest4068927-21kworker/u8:2+events_unbound23:28:352
3983733995243,5cyclictest4029250-21cron21:20:010
3983733995137,11cyclictest121ktimers/022:00:150
3983745995037,11cyclictest391ktimers/320:30:183
3983733994733,11cyclictest4091536-21sendmail00:20:000
3983733994635,8cyclictest440-21cron22:20:000
3983745994539,4cyclictest4039153-21kworker/u8:2+events_unbound21:48:193
3983745994539,4cyclictest4039153-21kworker/u8:2+events_unbound21:48:183
3983742994533,8cyclictest311ktimers/221:05:142
3983733994532,9cyclictest121ktimers/020:50:140
3983733994421,18cyclictest0-21swapper/000:05:010
3983733994416,23cyclictest4018571-21grep20:45:300
398373399441,11cyclictest4012679-21/usr/sbin/munin20:30:250
3983745994337,4cyclictest4047124-21kworker/u8:2+events_unbound22:13:193
3983745994333,6cyclictest4067439-21idleruntime-cro23:10:003
3983745994333,6cyclictest4067439-21idleruntime-cro23:10:003
3983737994332,8cyclictest4094931-21sh00:30:001
3983733994316,22cyclictest0-21swapper/021:55:270
398373399422,8cyclictest4033518-21/usr/sbin/munin21:30:240
3983733994214,25cyclictest4027644-21latency_hist21:15:010
398373399421,11cyclictest4080005-21sort23:45:170
398373399421,11cyclictest4060930-21/usr/sbin/munin22:50:170
3983745994135,4cyclictest4015317-21kworker/u8:1+events_unbound20:48:083
3983745994129,7cyclictest4023726-21grep21:00:263
3983745994121,5cyclictest131rcu_preempt23:15:183
3983742994129,9cyclictest311ktimers/221:45:152
3983742994129,9cyclictest311ktimers/221:45:142
3983737994134,4cyclictest4037806-21idleruntime-cro21:45:001
398373399418,7cyclictest4029609-21cat21:20:140
3983733994130,6cyclictest0-21swapper/023:20:010
398373399412,9cyclictest4057408-21fschecks_time22:40:160
398373399411,13cyclictest4037937-21latency_hist21:45:010
398373399411,12cyclictest4002118-21tr20:00:210
398373399411,10cyclictest4064002-21sh23:00:000
398373399411,10cyclictest3993574-21tr19:35:240
3983745994027,4cyclictest131rcu_preempt20:14:403
3983745994027,4cyclictest131rcu_preempt20:14:403
3983745994022,5cyclictest131rcu_preempt19:50:123
3983745994015,12cyclictest4079374-21kworker/u8:0-events_freezable_power_23:57:363
398373399408,5cyclictest4064599-21/usr/sbin/munin23:00:190
3983733994025,12cyclictest121ktimers/021:50:140
398373399402,12cyclictest4056792-21sendmail-msp22:40:000
398373399402,12cyclictest4056792-21sendmail-msp22:40:000
398373399401,8cyclictest4020963-21/usr/sbin/munin20:55:160
3983733994012,3cyclictest121ktimers/020:20:290
398373399401,17cyclictest4038034-21/usr/sbin/munin21:45:130
398373399401,17cyclictest4038034-21/usr/sbin/munin21:45:130
3983745993928,6cyclictest4058695-21sh22:45:003
3983745993924,5cyclictest131rcu_preempt23:25:153
3983745993919,3cyclictest131rcu_preempt22:15:343
3983742993932,4cyclictest4075950-21kworker/u8:3+events_unbound23:38:332
3983742993932,4cyclictest4075950-21kworker/u8:3+events_unbound23:38:332
3983737993931,4cyclictest0-21swapper/100:25:011
3983737993930,6cyclictest4096229-21unixbench_singl00:30:291
3983737993929,6cyclictest4026298-21fschecks_time21:10:151
3983737993920,6cyclictest4056457-21kworker/u8:0+events_unbound22:49:591
398373399398,5cyclictest4088303-21latency_hist00:10:010
398373399392,9cyclictest4060595-21cron22:50:000
398373399392,11cyclictest4016945-21sshd20:43:060
398373399391,9cyclictest3998835-21find19:50:240
398373399391,8cyclictest4046347-21latency_hist22:10:020
398373399391,5cyclictest4097899-21grep00:35:280
3983733993913,23cyclictest4093561-21cstates00:25:170
398373399391,11cyclictest4072282-21sed23:20:260
398373399391,10cyclictest4078743-21wc23:40:230
398373399391,10cyclictest3986322-21diskstats19:15:180
3983745993831,5cyclictest4009908-21kworker/u8:0+events_unbound20:38:073
3983737993830,5cyclictest3985455-21sh19:15:011
3983737993828,6cyclictest0-21swapper/123:15:001
398373399389,7cyclictest4035574-21cat21:35:250
398373399388,4cyclictest4031896-21sendmail_mailqu21:25:230
398373399383,7cyclictest4005117-21fschecks_count20:10:190
398373399383,7cyclictest4005117-21fschecks_count20:10:190
3983733993830,4cyclictest0-21swapper/023:10:000
3983733993830,4cyclictest0-21swapper/023:10:000
398373399382,6cyclictest4074067-21unixbench_singl23:25:290
398373399381,9cyclictest4084165-21wc23:55:260
398373399381,9cyclictest4051176-21tail22:20:270
398373399381,8cyclictest3988918-21grep19:20:260
398373399381,11cyclictest4023753-21users21:00:260
398373399381,10cyclictest4001342-21tr20:00:000
398373399381,10cyclictest3984443-21ntpq19:10:260
398373399380,13cyclictest0-21swapper/020:35:190
3983745993730,5cyclictest3997647-21kworker/u8:0+events_unbound19:47:583
3983745993726,6cyclictest0-21swapper/321:40:003
3983737993730,4cyclictest4008366-21munin-plugin-st20:20:011
3983737993729,4cyclictest0-21swapper/120:30:211
3983737993727,6cyclictest0-21swapper/121:50:271
3983737993727,6cyclictest0-21swapper/121:20:251
3983737993718,6cyclictest4047124-21kworker/u8:2+events_unbound22:36:141
3983737993718,6cyclictest4047124-21kworker/u8:2+events_unbound22:36:141
398373399376,5cyclictest0-21swapper/022:30:180
398373399375,6cyclictest3990250-21/usr/sbin/munin19:25:280
398373399373,10cyclictest4076399-21apt-get23:35:160
398373399373,10cyclictest4076399-21apt-get23:35:150
398373399372,7cyclictest4082570-21unixbench_singl23:50:260
3983733993712,2cyclictest0-21swapper/000:10:190
398373399371,10cyclictest3996665-21sort19:45:190
398373399370,14cyclictest0-21swapper/020:25:190
398374599363,3cyclictest131rcu_preempt22:45:133
3983745993627,4cyclictest0-21swapper/320:15:283
3983745993611,5cyclictest131rcu_preempt20:50:273
3983737993625,6cyclictest0-21swapper/120:25:281
3983737993622,8cyclictest0-21swapper/123:50:271
398373399361,9cyclictest4025844-21sh21:10:000
398373399361,6cyclictest4096235-21grep00:30:300
3983745993525,4cyclictest0-21swapper/321:55:233
3983745993522,9cyclictest391ktimers/322:20:143
3983737993529,4cyclictest3986091-21sendmail_mailst19:15:031
3983737993527,4cyclictest0-21swapper/121:40:011
3983737993523,7cyclictest0-21swapper/120:45:281
398373799351,16cyclictest24-21ksoftirqd/122:55:151
398373399359,3cyclictest4052281-21latency22:25:200
398373399356,7cyclictest4048306-21cut22:15:010
398373399351,6cyclictest630-21snmpd23:30:200
398373399351,13cyclictest4008352-21cron20:20:000
3983745993426,4cyclictest0-21swapper/323:45:263
3983745993426,4cyclictest0-21swapper/319:17:483
3983745993426,4cyclictest0-21swapper/319:15:013
3983745993425,5cyclictest4030237-21sendmail_mailtr21:20:243
3983745993423,4cyclictest131rcu_preempt22:27:083
3983745993419,5cyclictest131rcu_preempt22:30:133
3983745993416,4cyclictest4009908-21kworker/u8:0-events_freezable_power_20:23:503
3983745993411,11cyclictest0-21swapper/300:20:213
3983742993424,7cyclictest0-21swapper/223:05:242
3983742993424,7cyclictest0-21swapper/223:05:242
3983742993423,6cyclictest0-21swapper/222:15:002
3983737993425,5cyclictest3992355-21grep19:30:271
3983737993420,9cyclictest231ktimers/120:55:171
398373399341,9cyclictest4092173-21grep00:20:190
398374599339,11cyclictest40-21ksoftirqd/323:50:263
398374599338,3cyclictest131rcu_preempt19:30:263
3983745993323,6cyclictest0-21swapper/300:25:303
3983745993316,6cyclictest4079374-21kworker/u8:0+events_unbound00:15:383
3983745993316,5cyclictest131rcu_preempt22:10:023
3983745993315,4cyclictest131rcu_preempt23:10:593
3983742993327,4cyclictest3989330-21kworker/u8:2+events_unbound19:27:502
3983742993325,4cyclictest0-21swapper/220:02:592
3983737993327,4cyclictest4030594-21kworker/u8:0+events_unbound21:28:151
3983737993326,4cyclictest4058860-21latency_hist22:45:021
3983737993324,6cyclictest0-21swapper/123:20:001
398373399331,7cyclictest4068646-21unixbench_multi23:10:270
398373399331,10cyclictest3994854-21df19:40:170
398374599328,12cyclictest40-21ksoftirqd/319:55:293
3983745993224,6cyclictest4054700-21kworker/u8:1+events_unbound22:38:253
3983745993224,6cyclictest4054700-21kworker/u8:1+events_unbound22:38:253
3983745993224,4cyclictest0-21swapper/323:00:013
3983745993224,4cyclictest0-21swapper/300:00:293
3983745993222,6cyclictest4032209-21grep21:25:253
3983745993218,3cyclictest131rcu_preempt21:52:053
3983745993213,12cyclictest131rcu_preempt20:25:033
3983745993212,6cyclictest40-21ksoftirqd/319:45:023
3983742993225,3cyclictest0-21swapper/219:20:012
3983742993224,4cyclictest0-21swapper/222:00:192
3983742993224,4cyclictest0-21swapper/220:38:072
3983742993223,5cyclictest0-21swapper/222:55:292
3983742993222,6cyclictest0-21swapper/219:35:282
3983737993224,5cyclictest0-21swapper/121:35:011
3983737993224,4cyclictest0-21swapper/122:15:151
3983737993222,6cyclictest0-21swapper/120:40:281
3983737993221,6cyclictest0-21swapper/122:25:281
3983737993218,7cyclictest131rcu_preempt23:25:241
3983737993217,9cyclictest131rcu_preempt00:00:161
3983737993217,7cyclictest131rcu_preempt20:25:011
398373399325,8cyclictest4003534-21basename20:05:180
398373399321,7cyclictest3992788-21cron19:35:010
398374599319,5cyclictest131rcu_preempt23:35:003
3983745993124,5cyclictest391ktimers/321:05:143
3983745993113,4cyclictest131rcu_preempt22:50:283
3983745993112,6cyclictest131rcu_preempt23:40:203
3983745993111,5cyclictest131rcu_preempt20:05:193
3983745993110,12cyclictest131rcu_preempt20:40:273
3983742993123,4cyclictest0-21swapper/222:00:002
3983742993123,4cyclictest0-21swapper/219:50:172
3983737993124,5cyclictest4022340-21kworker/u8:2+events_unbound21:03:111
3983737993123,4cyclictest0-21swapper/120:05:271
3983737993122,4cyclictest0-21swapper/100:35:281
3983737993120,6cyclictest0-21swapper/100:15:321
3983737993119,6cyclictest0-21swapper/120:55:011
3983737993114,12cyclictest231ktimers/100:05:131
398374599307,15cyclictest40-21ksoftirqd/300:30:233
3983745993022,4cyclictest0-21swapper/300:05:273
3983745993014,5cyclictest131rcu_preempt23:20:223
3983745993011,3cyclictest131rcu_preempt00:37:193
3983745993010,12cyclictest40-21ksoftirqd/320:55:293
3983745993010,11cyclictest40-21ksoftirqd/321:15:273
3983742993023,4cyclictest0-21swapper/222:45:262
3983742993019,7cyclictest0-21swapper/223:15:272
3983737993022,6cyclictest3985284-21kworker/u8:1+events_unbound19:22:491
3983737993022,5cyclictest0-21swapper/123:45:261
3983737993019,6cyclictest0-21swapper/100:10:301
3983737993018,6cyclictest4025859-21kworker/u8:2+events_unbound21:19:531
3983737993016,4cyclictest3995907-21kworker/u8:1+events_unbound19:44:281
398374599297,7cyclictest40-21ksoftirqd/322:00:243
3983745992915,6cyclictest131rcu_preempt23:40:013
3983745992915,6cyclictest131rcu_preempt23:40:003
3983745992912,9cyclictest0-21swapper/300:13:033
3983745992911,5cyclictest131rcu_preempt19:20:203
3983745992911,13cyclictest0-21swapper/320:00:163
3983745992911,11cyclictest40-21ksoftirqd/321:10:233
3983745992911,11cyclictest40-21ksoftirqd/319:35:173
3983742992921,4cyclictest0-21swapper/222:40:012
3983737992922,5cyclictest3995907-21kworker/u8:1+events_unbound19:45:251
3983737992922,4cyclictest4001326-21cron20:00:011
3983737992921,4cyclictest0-21swapper/122:55:001
3983737992921,4cyclictest0-21swapper/100:00:011
398374599284,7cyclictest40-21ksoftirqd/323:00:173
3983745992810,10cyclictest0-21swapper/321:40:373
3983742992822,4cyclictest4056758-21apt-get22:40:012
3983742992821,5cyclictest4065487-21kworker/u8:0+events_unbound23:03:232
3983742992820,4cyclictest0-21swapper/222:25:282
3983742992820,4cyclictest0-21swapper/219:40:292
3983742992818,6cyclictest0-21swapper/222:05:262
3983742992817,8cyclictest311ktimers/200:30:152
3983737992820,5cyclictest0-21swapper/122:10:261
3983745992712,5cyclictest131rcu_preempt21:31:253
3983742992719,5cyclictest0-21swapper/222:35:002
3983742992718,6cyclictest4015786-21fschecks_count20:40:182
3983742992717,6cyclictest0-21swapper/220:50:172
3983737992712,4cyclictest0-21swapper/122:20:151
398373799271,12cyclictest0-21swapper/122:33:541
398374299268,4cyclictest4004647-21kworker/u8:1+events_unbound20:11:152
398374299268,4cyclictest4004647-21kworker/u8:1+events_unbound20:11:152
3983742992620,4cyclictest4058411-21kworker/u8:3+events_unbound22:53:262
3983742992618,5cyclictest0-21swapper/221:40:192
3983742992618,5cyclictest0-21swapper/200:25:002
3983742992617,6cyclictest0-21swapper/223:20:272
3983742992617,6cyclictest0-21swapper/221:30:012
3983742992617,5cyclictest0-21swapper/219:30:262
3983742992616,7cyclictest0-21swapper/221:35:252
3983742992616,6cyclictest0-21swapper/223:50:262
3983742992615,7cyclictest0-21swapper/221:10:242
3983737992620,4cyclictest4009908-21kworker/u8:0+events_unbound20:36:081
3983737992619,5cyclictest4068927-21kworker/u8:2+events_unbound23:20:221
3983737992619,5cyclictest4004647-21kworker/u8:1+events_unbound20:10:211
3983737992619,5cyclictest4004647-21kworker/u8:1+events_unbound20:10:211
3983737992618,6cyclictest3999365-21kworker/u8:2+events_unbound19:53:471
398373799261,6cyclictest0-21swapper/123:05:121
398373799261,6cyclictest0-21swapper/123:05:111
3983742992518,4cyclictest4056457-21kworker/u8:0+events_unbound22:41:072
3983742992515,6cyclictest0-21swapper/222:15:272
3983742992512,9cyclictest1-21systemd21:18:122
3983737992515,7cyclictest4076553-21tune2fs23:35:171
3983737992515,7cyclictest4076553-21tune2fs23:35:171
3983737992513,6cyclictest4020583-21kworker/u8:0+events_unbound21:07:431
398374299245,4cyclictest131rcu_preempt23:40:022
3983742992418,3cyclictest0-21swapper/220:00:002
3983742992417,4cyclictest0-21swapper/220:30:212
3983742992415,6cyclictest0-21swapper/220:20:282
398373799247,10cyclictest24-21ksoftirqd/123:00:231
3983737992418,4cyclictest3989330-21kworker/u8:2+events_unbound19:29:321
398374299231,4cyclictest131rcu_preempt22:20:142
3983742992311,8cyclictest0-21swapper/200:00:162
3983742992310,10cyclictest0-21swapper/200:10:212
398373799236,3cyclictest131rcu_preempt22:00:151
398373799232,13cyclictest0-21swapper/119:38:471
3983737992314,5cyclictest0-21swapper/120:00:151
3983737992312,8cyclictest215-21dbus-daemon23:43:341
3983742992216,3cyclictest0-21swapper/221:50:222
3983742992216,3cyclictest0-21swapper/219:20:252
3983742992215,5cyclictest4015317-21kworker/u8:1+events_unbound20:58:112
3983742992215,3cyclictest0-21swapper/223:55:262
398374299221,4cyclictest131rcu_preempt19:45:162
3983742992214,5cyclictest0-21swapper/220:45:292
3983742992214,4cyclictest0-21swapper/200:35:282
3983742992214,4cyclictest0-21swapper/200:25:182
3983742992211,3cyclictest131rcu_preempt20:25:192
398373799225,10cyclictest24-21ksoftirqd/121:45:241
398373799225,10cyclictest24-21ksoftirqd/121:45:241
3983737992212,4cyclictest4045303-21kworker/u8:1+events_unbound22:08:101
398374299219,3cyclictest131rcu_preempt00:15:152
3983742992114,4cyclictest0-21swapper/200:05:272
3983742992112,6cyclictest0-21swapper/223:45:262
3983742992112,6cyclictest0-21swapper/219:15:002
3983742992111,7cyclictest4003364-21awk20:05:152
3983742992110,3cyclictest131rcu_preempt23:10:162
3983742992015,3cyclictest4074692-21/usr/sbin/munin23:30:222
3983742992015,3cyclictest4022854-21fschecks_count21:00:172
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional