You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-09 - 10:45

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Phytec
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackbslot5s.osadl.org (updated Sun Nov 09, 2025 00:44:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2030345995625,26cyclictest0-21swapper/121:45:021
2030345995323,26cyclictest0-21swapper/123:50:011
2030353995237,12cyclictest391ktimers/321:40:153
2030345995115,9cyclictest2035992-21cron19:25:011
2030345994820,4cyclictest0-21swapper/121:05:171
2030345994634,7cyclictest0-21swapper/123:05:271
2030345994434,6cyclictest2075386-21grep21:15:291
2030353994335,6cyclictest2107192-21kworker/u8:2+events_unbound23:28:413
2030353994335,6cyclictest2107192-21kworker/u8:2+events_unbound23:28:413
2030353994333,7cyclictest2034295-21sendmail19:20:003
2030350994331,8cyclictest2044530-21munin-plugin-st19:50:012
2030353994231,8cyclictest391ktimers/321:55:113
2030353994231,8cyclictest391ktimers/321:55:103
2030353994230,9cyclictest391ktimers/322:55:133
2030345994233,6cyclictest2134730-21munin-plugin-st00:10:001
2030345994231,6cyclictest2137888-21unixbench_multi00:15:261
2030353994134,5cyclictest2080687-21kworker/u8:1+events_unbound21:48:153
2030353994134,5cyclictest2080687-21kworker/u8:1+events_unbound21:48:153
2030353994134,5cyclictest2042799-21kworker/u8:2+events_unbound19:47:523
2030345994132,4cyclictest0-21swapper/100:30:171
2030345994129,6cyclictest0-21swapper/123:20:261
2030341994131,6cyclictest2137872-21unixbench_multi00:15:250
2030345994030,6cyclictest2080585-21unixbench_singl21:30:261
2030345994029,7cyclictest2145172-21sendmail00:40:001
2030345994018,4cyclictest0-21swapper/100:04:591
2030345993930,4cyclictest0-21swapper/123:00:011
2030345993928,6cyclictest0-21swapper/120:35:261
2030341993930,4cyclictest0-21swapper/021:18:070
2030353993829,6cyclictest2071795-21unixbench_multi21:05:253
2030350993830,4cyclictest0-21swapper/220:15:012
2030350993827,7cyclictest0-21swapper/220:30:012
2030345993827,7cyclictest0-21swapper/121:50:271
2030341993820,15cyclictest121ktimers/022:27:490
2030341993820,15cyclictest121ktimers/022:27:480
2030353993730,4cyclictest2130871-21unixbench_singl23:55:273
2030350993724,9cyclictest2100393-21latency_hist22:30:012
2030350993724,9cyclictest2100393-21latency_hist22:30:012
2030345993727,6cyclictest0-21swapper/100:30:001
2030345993725,6cyclictest0-21swapper/119:30:191
2030341993729,4cyclictest0-21swapper/022:03:170
2030341993729,4cyclictest0-21swapper/022:03:160
2030341993712,17cyclictest131rcu_preempt23:35:180
2030353993628,5cyclictest2117662-21sh23:20:013
2030353993625,7cyclictest0-21swapper/319:22:483
2030353993621,12cyclictest2141994-21seq00:30:143
2030345993627,6cyclictest2108439-21unixbench_multi22:50:251
2030345993627,4cyclictest0-21swapper/122:20:011
2030341993628,5cyclictest121ktimers/000:05:130
2030341993627,4cyclictest2033809-21grep19:15:250
2030353993528,5cyclictest2050983-21kworker/u8:2+events_unbound20:12:553
2030353993527,4cyclictest0-21swapper/322:50:263
2030353993526,6cyclictest2041540-21fschecks_time19:40:173
2030353993525,6cyclictest0-21swapper/319:55:253
2030353993514,6cyclictest131rcu_preempt22:40:123
2030350993528,4cyclictest0-21swapper/223:20:262
2030350993527,4cyclictest0-21swapper/222:35:162
2030341993527,6cyclictest121ktimers/020:25:160
2030341993526,6cyclictest0-21swapper/020:50:300
2030341993525,7cyclictest440-21cron23:24:590
2030341993514,3cyclictest131rcu_preempt00:00:260
2030353993428,4cyclictest2039122-21kworker/u8:1+events_unbound19:52:523
2030353993427,4cyclictest0-21swapper/323:24:593
2030353993426,4cyclictest0-21swapper/321:25:013
2030353993425,5cyclictest0-21swapper/320:05:303
2030350993426,5cyclictest2086068-21unixbench_multi21:45:262
2030350993426,5cyclictest2086068-21unixbench_multi21:45:252
2030350993424,5cyclictest0-21swapper/200:40:012
2030350993423,8cyclictest2108510-21grep22:50:262
2030345993424,6cyclictest0-21swapper/120:15:281
2030345993423,6cyclictest2129177-21grep23:50:301
2030353993326,5cyclictest2073821-21kworker/u8:1+events_unbound21:16:423
2030353993322,7cyclictest0-21swapper/320:45:003
2030353993318,6cyclictest2077527-21kworker/u8:1+events_unbound21:25:113
2030350993322,7cyclictest0-21swapper/219:40:012
2030345993325,4cyclictest0-21swapper/122:23:201
2030345993325,4cyclictest0-21swapper/121:23:061
2030345993323,6cyclictest0-21swapper/122:29:591
2030345993323,6cyclictest0-21swapper/122:29:591
2030345993323,6cyclictest0-21swapper/120:05:281
203034199339,8cyclictest11-21ksoftirqd/020:40:000
203034199338,7cyclictest11-21ksoftirqd/023:20:000
2030341993323,5cyclictest0-21swapper/019:25:290
2030341993322,8cyclictest121ktimers/023:40:200
2030341993322,8cyclictest121ktimers/020:15:000
2030341993321,4cyclictest131rcu_preempt00:37:060
2030353993225,4cyclictest0-21swapper/321:10:173
2030353993224,5cyclictest2087830-21unixbench_singl21:50:273
2030353993224,4cyclictest0-21swapper/320:00:273
2030353993223,6cyclictest2082918-21sendmail21:39:593
2030353993223,4cyclictest0-21swapper/322:20:273
2030353993221,6cyclictest0-21swapper/320:50:303
2030350993220,8cyclictest2054975-21sendmail20:20:012
203034199329,7cyclictest11-21ksoftirqd/020:20:000
203034199329,5cyclictest131rcu_preempt20:40:260
203034199325,6cyclictest151rcuc/021:00:010
2030341993224,5cyclictest2081158-21apt-config21:35:000
2030341993221,7cyclictest121ktimers/022:35:200
2030341993216,4cyclictest131rcu_preempt21:12:360
2030341993215,4cyclictest121ktimers/022:51:510
2030341993212,5cyclictest131rcu_preempt22:15:210
203035399319,12cyclictest40-21ksoftirqd/320:38:003
2030353993124,5cyclictest2103712-21kworker/u8:0+events_unbound22:45:293
2030353993124,4cyclictest0-21swapper/322:00:173
2030353993124,4cyclictest0-21swapper/322:00:173
2030353993121,6cyclictest0-21swapper/322:25:293
2030353993121,6cyclictest0-21swapper/322:25:283
2030353993121,6cyclictest0-21swapper/320:20:003
2030353993119,8cyclictest0-21swapper/300:39:593
2030353993117,4cyclictest2107192-21kworker/u8:2+events_unbound23:47:403
2030350993123,5cyclictest0-21swapper/223:45:272
2030350993123,5cyclictest0-21swapper/222:00:012
2030350993123,5cyclictest0-21swapper/222:00:002
2030345993123,4cyclictest0-21swapper/121:40:001
2030345993123,4cyclictest0-21swapper/120:40:171
2030345993121,7cyclictest2086066-21grep21:45:251
2030345993121,7cyclictest2086066-21grep21:45:251
2030345993121,6cyclictest0-21swapper/122:40:261
203034199318,7cyclictest11-21ksoftirqd/022:45:000
2030341993120,8cyclictest121ktimers/021:00:220
2030341993120,7cyclictest151rcuc/000:28:370
2030341993119,9cyclictest121ktimers/021:25:010
2030341993118,9cyclictest121ktimers/020:00:130
2030341993118,4cyclictest131rcu_preempt23:10:180
2030341993116,4cyclictest131rcu_preempt19:55:140
2030341993113,9cyclictest131rcu_preempt23:56:020
2030341993112,12cyclictest131rcu_preempt19:35:210
2030341993112,11cyclictest131rcu_preempt23:05:250
2030341993112,11cyclictest131rcu_preempt00:10:050
2030341993111,3cyclictest131rcu_preempt23:45:230
2030341993110,5cyclictest131rcu_preempt22:30:230
2030341993110,5cyclictest131rcu_preempt21:55:270
2030341993110,5cyclictest131rcu_preempt21:55:260
203035399301,7cyclictest0-21swapper/300:00:123
2030353993010,12cyclictest40-21ksoftirqd/300:10:293
2030350993023,4cyclictest0-21swapper/223:05:262
2030350993023,4cyclictest0-21swapper/222:55:262
2030350993022,4cyclictest0-21swapper/223:00:152
2030350993021,5cyclictest0-21swapper/220:05:202
2030350993020,7cyclictest2075433-21head21:15:292
2030345993021,4cyclictest0-21swapper/120:20:271
2030345993020,6cyclictest0-21swapper/120:30:161
2030345993020,4cyclictest0-21swapper/119:20:011
203034199307,7cyclictest11-21ksoftirqd/020:25:010
203034199304,7cyclictest11-21ksoftirqd/021:45:160
203034199304,7cyclictest11-21ksoftirqd/021:45:160
203034199304,6cyclictest11-21ksoftirqd/023:25:180
203034199304,6cyclictest11-21ksoftirqd/023:25:170
203034199302,8cyclictest11-21ksoftirqd/000:25:010
2030341993023,4cyclictest0-21swapper/021:25:270
2030341993022,4cyclictest121ktimers/021:10:000
2030341993020,7cyclictest121ktimers/021:35:010
2030341993020,6cyclictest2106203-21cat22:45:200
2030341993019,8cyclictest121ktimers/021:50:240
2030341993017,6cyclictest2035641-21kworker/u8:1+events_unbound19:24:400
2030341993016,3cyclictest121ktimers/023:05:010
2030341993014,9cyclictest131rcu_preempt20:30:220
2030341993014,4cyclictest131rcu_preempt20:06:090
2030341993013,9cyclictest11-21ksoftirqd/019:55:010
2030341993013,10cyclictest131rcu_preempt19:40:200
2030341993012,15cyclictest121ktimers/022:05:060
2030341993011,8cyclictest131rcu_preempt23:50:170
2030341993010,17cyclictest121ktimers/022:55:200
203035399297,12cyclictest0-21swapper/319:30:313
203035399292,3cyclictest0-21swapper/319:25:153
2030353992922,5cyclictest2089910-21kworker/u8:0+events_unbound22:05:233
2030353992922,4cyclictest0-21swapper/323:35:003
2030353992922,4cyclictest0-21swapper/323:35:003
2030353992922,4cyclictest0-21swapper/323:00:253
2030353992921,4cyclictest0-21swapper/322:35:303
2030353992921,4cyclictest0-21swapper/322:10:253
2030353992918,6cyclictest2137870-21grep00:15:253
2030353992912,6cyclictest40-21ksoftirqd/323:55:003
2030350992920,5cyclictest0-21swapper/200:08:472
2030350992918,7cyclictest2062462-21uniq20:40:182
2030345992921,4cyclictest0-21swapper/120:00:011
2030345992919,6cyclictest0-21swapper/122:30:171
2030341992920,6cyclictest121ktimers/020:45:210
2030341992919,7cyclictest151rcuc/019:14:470
2030341992919,7cyclictest151rcuc/019:14:460
2030341992915,8cyclictest131rcu_preempt22:20:280
2030341992911,3cyclictest131rcu_preempt19:45:470
2030341992911,14cyclictest151rcuc/023:30:140
2030341992911,14cyclictest151rcuc/023:30:140
2030353992822,4cyclictest2094779-21kworker/u8:3+events_unbound22:18:203
2030353992819,7cyclictest2065231-21kworker/u8:1+events_unbound21:00:043
2030353992819,4cyclictest0-21swapper/319:35:313
2030350992821,5cyclictest311ktimers/219:59:252
2030350992821,4cyclictest2120671-21unixbench_singl23:25:262
2030350992821,4cyclictest2120671-21unixbench_singl23:25:252
2030350992821,4cyclictest2056262-21head20:20:272
2030350992820,5cyclictest0-21swapper/221:10:262
2030350992820,4cyclictest0-21swapper/222:15:282
2030350992819,6cyclictest0-21swapper/219:40:262
2030345992820,4cyclictest0-21swapper/122:50:001
2030345992818,6cyclictest0-21swapper/119:25:291
2030345992816,7cyclictest0-21swapper/123:10:291
2030341992818,7cyclictest121ktimers/000:35:010
2030341992812,5cyclictest131rcu_preempt22:10:240
203035399278,4cyclictest131rcu_preempt20:56:233
203035399277,6cyclictest131rcu_preempt20:25:193
203035399276,3cyclictest131rcu_preempt23:10:303
2030353992719,4cyclictest0-21swapper/323:40:003
2030353992716,7cyclictest0-21swapper/321:30:253
203035399271,5cyclictest40-21ksoftirqd/320:35:013
2030353992713,6cyclictest2030498-21kworker/u8:1+events_unbound19:11:263
2030353992713,6cyclictest2030498-21kworker/u8:1+events_unbound19:11:263
2030350992721,3cyclictest0-21swapper/222:40:252
2030350992719,5cyclictest0-21swapper/223:40:172
2030350992719,4cyclictest0-21swapper/200:20:002
2030345992720,4cyclictest2031978-21idleruntime-cro19:15:001
2030345992720,4cyclictest2031978-21idleruntime-cro19:15:001
2030345992720,3cyclictest2117654-21kthreadd23:19:591
2030345992718,6cyclictest0-21swapper/100:25:011
2030345992718,4cyclictest0-21swapper/122:00:261
2030345992718,4cyclictest0-21swapper/122:00:261
2030345992717,6cyclictest0-21swapper/121:55:241
2030345992717,6cyclictest0-21swapper/121:55:241
2030345992716,7cyclictest0-21swapper/121:25:271
203034199274,7cyclictest11-21ksoftirqd/021:40:210
2030353992610,7cyclictest40-21ksoftirqd/300:20:253
2030350992619,4cyclictest2045447-21fgrep19:50:252
2030350992619,4cyclictest0-21swapper/219:25:282
2030350992619,3cyclictest0-21swapper/223:35:002
2030350992619,3cyclictest0-21swapper/223:35:002
2030345992618,4cyclictest0-21swapper/122:35:181
2030345992616,6cyclictest0-21swapper/123:55:261
2030345992616,6cyclictest0-21swapper/120:45:201
2030345992616,6cyclictest0-21swapper/120:30:001
203035399258,6cyclictest40-21ksoftirqd/320:45:163
203035399257,6cyclictest40-21ksoftirqd/300:05:173
203035399256,11cyclictest40-21ksoftirqd/322:30:253
203035399252,4cyclictest131rcu_preempt23:05:163
2030353992519,4cyclictest2107192-21kworker/u8:2+events_unbound23:44:423
2030350992518,5cyclictest2130950-21kworker/u8:0+events_unbound00:22:492
2030350992518,3cyclictest0-21swapper/220:45:152
2030350992517,5cyclictest0-21swapper/221:45:012
2030350992517,4cyclictest0-21swapper/221:30:262
2030350992516,6cyclictest311ktimers/222:30:062
2030350992516,5cyclictest0-21swapper/223:50:222
2030345992516,5cyclictest0-21swapper/119:35:241
2030345992515,6cyclictest0-21swapper/120:50:301
203035399248,10cyclictest40-21ksoftirqd/320:20:053
2030350992419,2cyclictest0-21swapper/222:20:162
2030350992418,3cyclictest0-21swapper/220:00:262
2030350992418,3cyclictest0-21swapper/219:25:002
2030350992418,3cyclictest0-21swapper/219:15:032
2030350992417,5cyclictest311ktimers/222:00:262
2030350992417,5cyclictest311ktimers/222:00:252
2030350992417,4cyclictest2124544-21latency_hist23:40:012
2030345992415,5cyclictest0-21swapper/123:25:261
2030345992415,5cyclictest0-21swapper/123:25:261
2030345992415,5cyclictest0-21swapper/121:10:221
2030345992413,8cyclictest2045481-21mailstats19:50:271
2030350992317,3cyclictest0-21swapper/221:20:112
2030350992316,4cyclictest0-21swapper/221:00:242
2030350992316,4cyclictest0-21swapper/220:30:272
2030350992314,5cyclictest2061990-21cron20:40:012
2030350992310,4cyclictest2115937-21cron23:15:002
2030345992316,3cyclictest0-21swapper/100:15:011
2030345992315,4cyclictest0-21swapper/120:55:251
203035099221,6cyclictest20819052cat21:35:212
2030350992216,3cyclictest0-21swapper/223:55:252
2030350992216,3cyclictest0-21swapper/223:15:172
2030350992216,3cyclictest0-21swapper/200:25:182
2030350992215,4cyclictest2079296-21idleruntime-cro21:30:012
2030350992215,3cyclictest0-21swapper/222:50:022
2030350992213,5cyclictest440-21cron20:55:002
2030345992216,3cyclictest0-21swapper/123:35:211
2030345992214,5cyclictest2112542-21cron23:05:011
2030345992214,5cyclictest0-21swapper/120:15:011
2030350992114,5cyclictest311ktimers/222:05:162
2030350992113,6cyclictest311ktimers/219:31:012
2030345992115,3cyclictest0-21swapper/120:00:221
2030345992114,3cyclictest0-21swapper/119:40:251
2030345992113,5cyclictest231ktimers/119:47:581
2030350992014,3cyclictest0-21swapper/200:10:282
2030350992013,5cyclictest311ktimers/200:30:172
2030350992010,7cyclictest2071009-21fw_packets21:05:172
2030345992014,4cyclictest0-21swapper/121:00:211
2030345992014,3cyclictest0-21swapper/122:10:211
2030345992010,7cyclictest2122257-21cat23:30:261
2030345992010,7cyclictest2122257-21cat23:30:251
2030350991914,3cyclictest637-21runrttasks22:10:202
2030350991913,3cyclictest2067349-21/usr/sbin/munin20:55:162
2030345991911,5cyclictest0-21swapper/122:05:231
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional