You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-05-12 - 07:11

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Phytec
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot5s.osadl.org (updated Mon May 12, 2025 00:44:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
781677995142,5cyclictest812991-21cut20:40:000
781680995041,5cyclictest867756-21munin-plugin-st23:20:001
781680994938,7cyclictest857110-21grep22:45:311
781685994739,5cyclictest891745-21ls00:30:002
781677994732,12cyclictest121ktimers/000:35:120
781685994639,4cyclictest872835-21cron23:35:002
781689994538,4cyclictest888343-21munin-plugin-st00:20:003
781689994335,4cyclictest840391-21sendmail22:00:013
781680994331,8cyclictest845278-21grep22:10:271
781677994331,9cyclictest121ktimers/019:25:140
781677994323,17cyclictest0-21swapper/022:00:010
781677994126,6cyclictest131rcu_preempt23:05:250
781685994032,4cyclictest895128-21sh00:40:002
781685994028,10cyclictest311ktimers/222:35:132
781685994025,8cyclictest131rcu_preempt19:55:272
781680994030,7cyclictest847377-21munin-run22:20:001
781680993931,4cyclictest0-21swapper/121:00:221
781680993930,5cyclictest863951-21unixbench_singl23:05:251
781680993928,9cyclictest231ktimers/121:50:151
781689993825,10cyclictest391ktimers/320:55:163
781685993821,11cyclictest804196-21kworker/u8:2+events_unbound20:15:322
781680993826,9cyclictest231ktimers/119:30:121
78167799388,22cyclictest844107-21sed22:10:020
781677993831,4cyclictest0-21swapper/019:55:010
781677993828,7cyclictest828092-21grep21:20:240
781677993818,5cyclictest0-21swapper/020:25:230
781685993727,6cyclictest850777-21auditctl22:30:002
781685993726,4cyclictest131rcu_preempt21:21:342
781685993719,11cyclictest835077-21kworker/u8:2-events_freezable_power_22:00:362
781685993717,17cyclictest311ktimers/223:09:222
781680993711,5cyclictest24-21ksoftirqd/100:10:061
781689993628,5cyclictest867750-21munin-plugin-st23:20:003
781685993626,6cyclictest0-21swapper/223:05:002
781685993624,4cyclictest131rcu_preempt23:46:402
781685993619,10cyclictest829844-21kworker/u8:0+events_unbound21:39:032
781685993618,4cyclictest131rcu_preempt21:05:252
781685993617,4cyclictest0-21swapper/220:55:242
781685993612,12cyclictest131rcu_preempt22:05:252
781677993623,4cyclictest131rcu_preempt20:05:240
781677993622,5cyclictest131rcu_preempt20:32:570
781677993622,5cyclictest131rcu_preempt20:32:570
781677993620,8cyclictest131rcu_preempt22:20:440
781677993620,8cyclictest131rcu_preempt22:20:440
781685993524,7cyclictest0-21swapper/219:30:252
781685993519,4cyclictest0-21swapper/221:57:452
781685993518,5cyclictest53-21kcompactd019:12:102
781685993516,10cyclictest131rcu_preempt21:15:212
781685993515,9cyclictest0-21swapper/219:35:302
781685993514,10cyclictest131rcu_preempt19:45:142
781680993521,3cyclictest131rcu_preempt20:26:111
781677993529,4cyclictest845679-21cron22:15:000
781677993527,4cyclictest0-21swapper/019:15:020
781677993523,6cyclictest131rcu_preempt21:51:150
781677993523,6cyclictest131rcu_preempt00:16:210
781677993517,9cyclictest131rcu_preempt19:30:120
781677993514,11cyclictest131rcu_preempt00:30:010
781689993427,3cyclictest0-21swapper/323:55:003
781689993425,4cyclictest0-21swapper/319:15:023
781689993424,7cyclictest861341-21fschecks_count23:00:173
78168599345,14cyclictest311ktimers/220:35:262
781685993424,6cyclictest0-21swapper/219:15:052
781685993423,6cyclictest0-21swapper/220:30:162
781685993423,6cyclictest0-21swapper/220:30:162
781685993422,6cyclictest794547-21kworker/u8:1+events_unbound19:53:562
781685993422,5cyclictest131rcu_preempt23:50:262
781685993420,8cyclictest884489-21kworker/u8:0+events_unbound00:18:422
781685993420,5cyclictest807550-21kworker/u8:3+events_unbound20:21:322
781685993419,5cyclictest131rcu_preempt19:40:222
78168599341,8cyclictest32-21ksoftirqd/200:05:142
781685993418,5cyclictest131rcu_preempt22:25:002
781685993418,5cyclictest131rcu_preempt22:25:002
781685993418,10cyclictest864299-21kworker/u8:1+events_unbound23:14:282
781685993416,5cyclictest131rcu_preempt21:30:012
781685993415,11cyclictest131rcu_preempt23:20:002
781685993413,13cyclictest131rcu_preempt19:25:242
781685993410,8cyclictest32-21ksoftirqd/221:40:262
78168099345,15cyclictest24-21ksoftirqd/122:40:261
781680993426,4cyclictest0-21swapper/119:45:011
781680993425,4cyclictest0-21swapper/123:35:281
781680993425,4cyclictest0-21swapper/123:35:271
781680993424,6cyclictest0-21swapper/120:10:001
781680993423,7cyclictest0-21swapper/100:35:251
781680993422,9cyclictest231ktimers/119:10:131
781677993427,5cyclictest151rcuc/022:58:280
781677993422,6cyclictest131rcu_preempt20:24:330
781677993419,9cyclictest131rcu_preempt19:35:280
781677993418,10cyclictest131rcu_preempt00:20:210
781677993417,11cyclictest131rcu_preempt21:30:180
781677993417,11cyclictest131rcu_preempt21:30:180
781677993414,10cyclictest131rcu_preempt19:20:210
781677993411,14cyclictest846988-1kworker/0:1H+kblockd22:20:010
78168999332,6cyclictest131rcu_preempt00:25:123
781685993325,5cyclictest836979-21cron21:50:002
781685993325,4cyclictest0-21swapper/223:55:172
781685993323,6cyclictest0-21swapper/221:00:302
781685993323,5cyclictest0-21swapper/219:20:262
781685993323,4cyclictest852428-21kworker/u8:1+events_unbound22:54:112
781685993322,5cyclictest884489-21kworker/u8:0+events_unbound00:23:562
781685993318,9cyclictest884489-21kworker/u8:0+events_unbound00:13:482
781685993317,5cyclictest131rcu_preempt21:35:002
781685993317,5cyclictest131rcu_preempt21:35:002
781685993316,8cyclictest131rcu_preempt22:10:222
781685993316,11cyclictest864299-21kworker/u8:1+events_unbound23:35:362
781685993316,11cyclictest864299-21kworker/u8:1+events_unbound23:35:362
781685993315,8cyclictest131rcu_preempt22:55:132
781685993313,9cyclictest131rcu_preempt22:15:082
781680993324,4cyclictest0-21swapper/119:50:291
781680993323,6cyclictest872432-21grep23:30:271
781680993322,7cyclictest858853-21grep22:50:291
781680993322,7cyclictest814265-21grep20:40:301
781680993317,9cyclictest781670-21cyclictest21:25:211
78167799337,4cyclictest131rcu_preempt00:05:430
78167799337,4cyclictest121ktimers/020:00:160
781677993325,4cyclictest0-21swapper/020:00:010
781677993321,7cyclictest819342-21grep20:55:290
781677993318,10cyclictest131rcu_preempt23:50:190
781677993317,10cyclictest131rcu_preempt00:00:170
781677993313,12cyclictest131rcu_preempt23:40:250
781677993313,12cyclictest131rcu_preempt23:40:240
78167799331,19cyclictest0-21swapper/023:00:230
781689993223,4cyclictest0-21swapper/322:55:273
781685993223,4cyclictest843883-21kworker/u8:2+events_unbound22:31:422
781685993221,8cyclictest301rcuc/222:46:052
781685993219,7cyclictest789108-21kworker/u8:2+events_unbound20:00:352
781685993217,9cyclictest867495-21kworker/u8:0+events_unbound23:25:032
781685993217,8cyclictest131rcu_preempt23:40:242
781685993217,8cyclictest131rcu_preempt23:40:242
781685993214,5cyclictest131rcu_preempt00:30:282
781685993212,3cyclictest0-21swapper/223:20:262
781685993212,11cyclictest131rcu_preempt20:10:192
781685993212,10cyclictest0-21swapper/220:50:052
781680993224,4cyclictest0-21swapper/121:30:301
781680993224,4cyclictest0-21swapper/121:30:291
78167799325,16cyclictest11-21ksoftirqd/020:15:250
781677993226,3cyclictest0-21swapper/022:30:000
781677993221,5cyclictest131rcu_preempt19:19:410
781677993216,8cyclictest131rcu_preempt20:13:450
781677993215,15cyclictest121ktimers/021:40:200
781677993213,5cyclictest131rcu_preempt22:50:010
781677993213,11cyclictest131rcu_preempt20:55:010
781677993211,8cyclictest131rcu_preempt00:10:020
78168999312,6cyclictest131rcu_preempt21:20:113
781689993123,4cyclictest0-21swapper/322:05:163
781689993116,12cyclictest781670-21cyclictest23:30:133
781685993125,4cyclictest799024-21kworker/u8:0+events_unbound20:25:422
781685993123,4cyclictest0-21swapper/220:40:292
781685993122,4cyclictest131rcu_preempt22:40:172
781685993121,7cyclictest301rcuc/200:03:192
781685993121,6cyclictest0-21swapper/220:50:012
781685993112,11cyclictest131rcu_preempt20:05:202
78168099318,6cyclictest832723-21proc_pri21:35:231
78168099316,6cyclictest0-21swapper/100:30:191
781680993124,3cyclictest0-21swapper/120:55:291
781680993123,4cyclictest0-21swapper/123:25:191
781680993123,4cyclictest0-21swapper/120:35:181
781680993123,4cyclictest0-21swapper/120:15:001
781680993122,6cyclictest826591-21tr21:20:001
781680993121,7cyclictest861360-21tune2fs23:00:171
781680993114,8cyclictest787391-21kworker/u8:1-events_freezable_power_19:26:441
781680993112,6cyclictest783996-21ls19:15:031
78167799317,7cyclictest11-21ksoftirqd/020:50:010
78167799316,12cyclictest11-21ksoftirqd/019:47:230
78167799314,16cyclictest11-21ksoftirqd/021:35:150
781677993125,3cyclictest0-21swapper/022:40:000
781677993119,4cyclictest131rcu_preempt23:25:000
781677993111,5cyclictest131rcu_preempt21:25:230
781677993110,14cyclictest131rcu_preempt21:05:010
781689993022,4cyclictest881356-21sendmail00:00:003
781685993012,10cyclictest131rcu_preempt21:10:252
78168099306,6cyclictest0-21swapper/123:10:181
78168099304,6cyclictest24-21ksoftirqd/120:35:001
78168099304,6cyclictest24-21ksoftirqd/120:35:001
781680993022,4cyclictest0-21swapper/120:20:181
781680993021,6cyclictest0-21swapper/100:05:271
781680993014,9cyclictest131rcu_preempt22:33:591
781677993023,4cyclictest0-21swapper/021:15:230
781677993021,5cyclictest865654-21grep23:10:260
781677993019,8cyclictest121ktimers/021:10:300
781677993016,5cyclictest131rcu_preempt23:47:450
781677993015,9cyclictest131rcu_preempt23:15:140
781677993014,3cyclictest131rcu_preempt00:33:570
781677993011,7cyclictest11-21ksoftirqd/020:45:010
781677993011,5cyclictest131rcu_preempt23:30:000
781677993010,11cyclictest131rcu_preempt19:10:020
781689992923,4cyclictest840108-21kworker/u8:0+events_unbound22:13:423
781689992923,3cyclictest0-21swapper/319:25:003
781689992922,4cyclictest0-21swapper/323:10:263
781689992921,4cyclictest0-21swapper/321:20:013
781689992921,4cyclictest0-21swapper/320:25:173
78168099299,4cyclictest131rcu_preempt21:20:021
78168099298,12cyclictest131rcu_preempt19:20:171
78168099297,11cyclictest131rcu_preempt22:04:311
78168099296,5cyclictest24-21ksoftirqd/100:05:001
78168099296,13cyclictest131rcu_preempt19:57:091
78168099295,5cyclictest24-21ksoftirqd/123:40:261
78168099295,5cyclictest24-21ksoftirqd/123:40:261
78168099293,7cyclictest24-21ksoftirqd/121:45:241
781680992921,4cyclictest0-21swapper/100:20:181
78167799299,8cyclictest0-21swapper/021:46:240
78167799294,5cyclictest11-21ksoftirqd/022:05:010
78167799293,6cyclictest11-21ksoftirqd/023:30:220
781677992912,5cyclictest131rcu_preempt23:40:000
781677992912,5cyclictest131rcu_preempt23:40:000
781677992912,4cyclictest131rcu_preempt00:00:010
781677992911,4cyclictest131rcu_preempt22:40:210
78168999285,3cyclictest131rcu_preempt19:40:273
781689992820,4cyclictest0-21swapper/320:50:303
78168999281,4cyclictest131rcu_preempt22:35:133
78168099288,11cyclictest131rcu_preempt22:35:071
78168099286,6cyclictest24-21ksoftirqd/122:05:261
78168099285,5cyclictest24-21ksoftirqd/119:40:011
78168099284,7cyclictest24-21ksoftirqd/120:50:211
78168099283,7cyclictest0-21swapper/123:45:151
781680992822,3cyclictest0-21swapper/100:17:001
781680992820,4cyclictest0-21swapper/100:30:011
781680992818,6cyclictest795546-21grep19:45:241
781680992811,9cyclictest131rcu_preempt20:04:521
781680992811,4cyclictest131rcu_preempt21:40:341
78167799286,7cyclictest11-21ksoftirqd/021:10:010
781677992811,4cyclictest131rcu_preempt19:42:360
781689992719,6cyclictest799024-21kworker/u8:0+events_unbound20:05:123
781689992719,4cyclictest0-21swapper/319:55:263
781689992718,4cyclictest0-21swapper/300:40:013
781689992717,6cyclictest0-21swapper/300:10:013
781689992716,8cyclictest781670-21cyclictest20:15:143
78168099276,6cyclictest24-21ksoftirqd/120:45:151
78168099273,5cyclictest24-21ksoftirqd/123:20:161
78168099272,7cyclictest24-21ksoftirqd/122:55:151
781680992717,7cyclictest231ktimers/122:24:021
781680992717,7cyclictest231ktimers/122:24:011
781680992714,9cyclictest881361-21basename00:00:011
78168099271,14cyclictest24-21ksoftirqd/120:15:411
781677992711,5cyclictest131rcu_preempt22:30:210
781677992710,5cyclictest131rcu_preempt22:50:160
78168999262,4cyclictest862897-21apt-get23:05:123
781689992619,4cyclictest835077-21kworker/u8:2+events_unbound22:01:093
781689992618,4cyclictest0-21swapper/321:50:293
781689992616,5cyclictest801093-21cron20:05:003
781689992615,4cyclictest131rcu_preempt20:37:513
78168099265,5cyclictest24-21ksoftirqd/123:50:161
78168099262,7cyclictest24-21ksoftirqd/122:25:261
781680992617,7cyclictest231ktimers/121:59:031
781689992519,3cyclictest857571-21kthreadd22:50:003
781689992518,5cyclictest829844-21kworker/u8:0+events_unbound21:31:053
781689992518,5cyclictest829844-21kworker/u8:0+events_unbound21:31:053
781689992517,6cyclictest146-21jbd2/mmcblk1p2-821:36:053
781689992517,4cyclictest0-21swapper/300:00:283
781689992515,5cyclictest850336-21grep22:25:273
781689992514,8cyclictest796359-21diskstats19:50:193
78168099258,4cyclictest131rcu_preempt21:08:141
781680992516,6cyclictest231ktimers/121:10:291
78168999245,5cyclictest0-21swapper/320:34:113
78168999245,5cyclictest0-21swapper/320:34:103
781689992418,4cyclictest829844-21kworker/u8:0+events_unbound21:27:463
781689992415,4cyclictest440-21cron21:45:003
781689992317,4cyclictest789108-21kworker/u8:2+events_unbound19:45:193
781689992317,3cyclictest0-21swapper/323:20:163
781689992317,3cyclictest0-21swapper/322:45:013
781689992317,3cyclictest0-21swapper/321:10:203
781689992317,3cyclictest0-21swapper/319:15:023
781689992317,3cyclictest0-21swapper/300:20:163
781689992316,3cyclictest0-21swapper/300:10:313
781689992315,6cyclictest833328-21kworker/u8:1+events_unbound21:49:073
781689992315,5cyclictest0-21swapper/323:45:283
781689992315,4cyclictest0-21swapper/319:25:263
781689992313,7cyclictest822292-21ntp_states21:05:273
781689992313,6cyclictest391ktimers/322:15:193
781689992311,7cyclictest874556-21sendmail23:40:003
781689992311,7cyclictest874556-21sendmail23:40:003
78168999229,4cyclictest131rcu_preempt20:10:213
78168999229,3cyclictest0-21swapper/320:23:053
78168999222,9cyclictest0-21swapper/321:00:173
781689992215,3cyclictest0-21swapper/319:35:243
781689992214,5cyclictest875789-21unixbench_multi23:40:283
781689992214,5cyclictest875789-21unixbench_multi23:40:283
781689992214,5cyclictest3856880-21kworker/3:2-mm_percpu_wq00:30:083
781689992212,5cyclictest0-21swapper/323:25:293
781689992210,9cyclictest851122-21df_inode22:30:183
781689992114,5cyclictest855762-21kworker/u8:2+events_unbound22:52:133
781689992113,5cyclictest0-21swapper/320:45:173
781689992014,4cyclictest843883-21kworker/u8:2+events_unbound22:20:073
781689992014,4cyclictest843883-21kworker/u8:2+events_unbound22:20:073
781689992013,5cyclictest809577-21kworker/u8:1+events_unbound20:40:123
781689992013,4cyclictest763594-21kworker/u8:0+events_unbound19:30:513
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional