You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-06 - 12:31

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Phytec
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot5s.osadl.org (updated Mon Apr 06, 2026 00:44:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
252432994943,4cyclictest341973-21kworker/u8:1+events_unbound23:31:353
252427994935,9cyclictest256379-21sendmail19:20:012
252424994839,5cyclictest311390-21grep22:00:001
252432994741,4cyclictest318060-21kworker/u8:3+events_unbound22:51:263
252419994740,5cyclictest275074-21kworker/u8:1+events_unbound20:15:540
252432994639,5cyclictest321527-21kworker/u8:1+events_unbound22:31:213
252432994638,4cyclictest311379-21sh22:00:003
252424994623,7cyclictest131rcu_preempt20:50:121
252424994623,7cyclictest131rcu_preempt20:50:121
252419994611,7cyclictest131rcu_preempt19:55:110
252432994539,4cyclictest312890-21kworker/u8:2+events_unbound22:21:193
252432994438,4cyclictest281644-21kworker/u8:3+events_unbound22:01:143
252432994437,5cyclictest288757-21kworker/u8:0+events_unbound21:01:023
252424994436,5cyclictest326563-21kworker/u8:1+events_unbound23:01:281
252432994337,4cyclictest354148-21kworker/u8:3+events_unbound00:18:413
252432994337,4cyclictest311141-21kworker/u8:0+events_unbound22:06:153
252424994335,4cyclictest312693-21unixbench_singl22:00:291
252419994335,6cyclictest281644-21kworker/u8:3+events_unbound21:26:080
252432994234,4cyclictest0-21swapper/323:00:003
252432994218,11cyclictest131rcu_preempt23:16:063
252419994224,4cyclictest131rcu_preempt21:16:020
252419994222,15cyclictest0-21swapper/000:10:000
252432994134,5cyclictest280197-21kworker/u8:1+events_unbound20:30:563
252432994132,7cyclictest299206-21kworker/u8:2+events_unbound21:36:113
252432994132,5cyclictest318060-21kworker/u8:3+events_unbound23:06:283
252424994132,4cyclictest0-21swapper/123:41:351
252424994131,7cyclictest306063-21cron21:45:001
252419994135,4cyclictest336769-21kworker/u8:3+events_unbound23:16:320
252419994131,7cyclictest330147-21cron22:55:010
252419994127,10cyclictest121ktimers/022:35:120
252432994025,5cyclictest131rcu_preempt20:20:463
252432994022,11cyclictest131rcu_preempt00:35:283
252432994021,12cyclictest361163-21kworker/u8:2-events_freezable_power_00:31:343
252432994020,14cyclictest131rcu_preempt19:44:113
252419994015,20cyclictest131rcu_preempt20:25:020
252432993933,4cyclictest236584-21kworker/u8:2+events_unbound19:20:423
252432993933,4cyclictest236584-21kworker/u8:2+events_unbound19:20:423
252432993932,5cyclictest340228-21kworker/u8:0+events_unbound00:08:133
252432993932,5cyclictest312890-21kworker/u8:2+events_unbound22:26:203
252432993931,4cyclictest0-21swapper/322:15:153
252432993919,9cyclictest131rcu_preempt21:44:273
252432993919,10cyclictest131rcu_preempt22:42:203
252432993919,10cyclictest131rcu_preempt22:42:203
252432993910,7cyclictest40-21ksoftirqd/321:35:013
252427993932,4cyclictest305848-21kworker/u8:1+events_unbound21:46:122
252424993931,5cyclictest276850-21munin-plugin-st20:20:011
252424993929,6cyclictest0-21swapper/121:35:001
252424993927,7cyclictest0-21swapper/121:45:261
252419993930,6cyclictest267883-21unixbench_singl19:50:250
252419993930,6cyclictest267883-21unixbench_singl19:50:240
252419993928,7cyclictest307351-21grep21:45:250
252419993916,17cyclictest131rcu_preempt19:35:170
252432993818,11cyclictest288757-21kworker/u8:0+events_unbound20:53:373
252432993818,11cyclictest288757-21kworker/u8:0+events_unbound20:53:363
252432993818,10cyclictest269848-21kworker/u8:3+events_unbound20:03:433
252432993816,5cyclictest131rcu_preempt23:15:013
252432993814,11cyclictest131rcu_preempt20:42:003
252427993826,8cyclictest347215-21auditctl23:45:002
252424993827,7cyclictest0-21swapper/121:25:261
252424993826,7cyclictest0-21swapper/122:05:251
252419993821,5cyclictest131rcu_preempt00:00:240
252432993724,5cyclictest131rcu_preempt20:25:183
252432993721,5cyclictest131rcu_preempt19:50:253
252432993721,5cyclictest131rcu_preempt19:50:243
252432993720,4cyclictest131rcu_preempt21:14:023
252432993718,4cyclictest131rcu_preempt21:26:453
252432993718,11cyclictest131rcu_preempt21:45:253
252432993716,11cyclictest131rcu_preempt00:28:593
252427993729,4cyclictest0-21swapper/222:59:592
252424993728,4cyclictest0-21swapper/119:20:161
252424993728,4cyclictest0-21swapper/119:20:151
252424993726,7cyclictest330149-21auditctl22:55:001
252424993726,7cyclictest0-21swapper/100:04:591
252419993729,6cyclictest348617-21kworker/u8:1+events_unbound23:52:350
252419993729,4cyclictest0-21swapper/020:10:000
252419993728,5cyclictest345486-21cron23:40:000
252419993720,11cyclictest131rcu_preempt20:35:270
252419993719,9cyclictest131rcu_preempt00:38:120
252419993719,3cyclictest131rcu_preempt22:55:200
252419993716,10cyclictest131rcu_preempt21:50:250
252432993628,6cyclictest350457-21kworker/u8:2+events_unbound23:57:453
252432993627,7cyclictest391ktimers/322:45:093
252432993627,6cyclictest293651-21grep21:05:253
252432993625,7cyclictest345011-21grep23:35:243
252432993618,9cyclictest131rcu_preempt20:11:423
252432993616,12cyclictest131rcu_preempt23:02:593
252432993616,10cyclictest131rcu_preempt19:10:113
252424993626,6cyclictest0-21swapper/123:20:241
252424993626,6cyclictest0-21swapper/121:20:011
252424993621,6cyclictest264560-21kworker/u8:0+events_unbound20:11:131
252424993621,5cyclictest326563-21kworker/u8:1+events_unbound23:06:421
252424993618,6cyclictest340228-21kworker/u8:0+events_unbound23:28:221
252424993617,5cyclictest326563-21kworker/u8:1+events_unbound23:15:501
252419993628,4cyclictest310978-21head21:55:240
252419993626,6cyclictest0-21swapper/020:30:260
252419993620,5cyclictest131rcu_preempt00:15:000
252419993618,9cyclictest131rcu_preempt20:25:170
252419993616,3cyclictest0-21swapper/020:43:230
252419993615,5cyclictest131rcu_preempt23:15:010
252432993520,4cyclictest131rcu_preempt20:18:213
252432993517,4cyclictest0-21swapper/323:27:363
252432993515,11cyclictest131rcu_preempt22:12:113
252432993515,10cyclictest131rcu_preempt19:37:533
252432993514,5cyclictest131rcu_preempt19:18:473
252432993514,11cyclictest131rcu_preempt19:32:443
252427993524,7cyclictest0-21swapper/223:10:002
25242499355,7cyclictest24-21ksoftirqd/119:35:251
252424993526,5cyclictest0-21swapper/100:20:001
252424993524,5cyclictest0-21swapper/123:10:251
252424993522,7cyclictest0-21swapper/120:00:241
252424993518,6cyclictest301170-21kworker/u8:1+events_unbound21:36:541
25241999359,4cyclictest131rcu_preempt22:33:020
25241999357,20cyclictest11-21ksoftirqd/023:45:000
252419993529,4cyclictest269848-21kworker/u8:3+events_unbound20:10:520
252419993525,6cyclictest0-21swapper/021:10:010
252419993522,4cyclictest131rcu_preempt23:06:470
252419993520,9cyclictest131rcu_preempt00:25:220
252419993520,5cyclictest131rcu_preempt21:00:160
252419993519,5cyclictest131rcu_preempt19:15:020
252419993519,5cyclictest131rcu_preempt19:10:220
25243299349,13cyclictest40-21ksoftirqd/323:40:243
25243299346,18cyclictest40-21ksoftirqd/323:20:193
252432993426,4cyclictest0-21swapper/321:15:173
252432993419,5cyclictest280197-21kworker/u8:1+events_unbound20:36:333
252432993418,6cyclictest312890-21kworker/u8:2+events_unbound22:38:163
252432993418,4cyclictest131rcu_preempt23:46:223
252432993417,4cyclictest131rcu_preempt21:54:053
252432993416,10cyclictest131rcu_preempt21:20:263
252432993415,5cyclictest131rcu_preempt19:25:223
252432993415,5cyclictest131rcu_preempt19:25:223
252432993413,5cyclictest131rcu_preempt00:05:003
252427993426,4cyclictest0-21swapper/219:40:002
252427993424,6cyclictest0-21swapper/220:55:252
252427993424,6cyclictest0-21swapper/220:55:242
252424993424,6cyclictest0-21swapper/122:10:241
252424993424,6cyclictest0-21swapper/121:05:251
252424993424,6cyclictest0-21swapper/100:40:001
252424993423,6cyclictest0-21swapper/123:59:591
252424993418,7cyclictest338497-21kworker/u8:2+events_unbound23:30:071
252419993425,6cyclictest0-21swapper/020:05:000
252419993418,5cyclictest131rcu_preempt22:45:210
252419993417,11cyclictest131rcu_preempt21:14:110
252419993415,9cyclictest131rcu_preempt22:10:200
252419993415,5cyclictest131rcu_preempt21:44:530
252419993415,4cyclictest131rcu_preempt23:55:200
252432993317,4cyclictest131rcu_preempt19:45:243
252432993315,5cyclictest131rcu_preempt00:10:223
252432993315,4cyclictest131rcu_preempt20:05:243
252432993311,5cyclictest131rcu_preempt20:55:213
252432993311,5cyclictest131rcu_preempt20:55:213
252427993324,5cyclictest312688-21unixbench_singl22:00:292
252424993325,4cyclictest0-21swapper/123:55:001
252424993324,5cyclictest0-21swapper/100:35:001
252424993323,5cyclictest0-21swapper/120:25:161
252424993322,7cyclictest0-21swapper/120:00:001
252424993322,6cyclictest0-21swapper/100:05:231
252424993318,9cyclictest131rcu_preempt22:35:001
252424993316,5cyclictest271628-21kworker/u8:2+events_unbound20:09:121
25241999338,5cyclictest131rcu_preempt00:15:100
25241999332,7cyclictest11-21ksoftirqd/022:05:120
252419993318,10cyclictest131rcu_preempt22:25:170
252419993317,5cyclictest131rcu_preempt23:20:120
252419993317,5cyclictest131rcu_preempt23:00:230
252419993317,5cyclictest131rcu_preempt22:20:190
252419993317,10cyclictest131rcu_preempt21:35:240
252419993317,10cyclictest131rcu_preempt20:50:160
252419993317,10cyclictest131rcu_preempt20:50:160
252419993316,5cyclictest131rcu_preempt21:30:150
252419993316,5cyclictest131rcu_preempt00:30:240
252419993315,13cyclictest131rcu_preempt19:40:160
252419993314,10cyclictest131rcu_preempt19:45:140
252419993313,12cyclictest131rcu_preempt19:30:200
25243299329,12cyclictest40-21ksoftirqd/320:45:223
252432993217,5cyclictest131rcu_preempt19:59:413
252432993215,4cyclictest131rcu_preempt23:50:413
252427993224,4cyclictest0-21swapper/220:35:272
252424993224,4cyclictest0-21swapper/122:46:241
252424993222,5cyclictest345503-21cut23:40:011
252424993221,7cyclictest0-21swapper/121:10:251
252424993217,6cyclictest318060-21kworker/u8:3+events_unbound22:58:011
252424993213,6cyclictest131rcu_preempt19:10:111
25241999324,14cyclictest11-21ksoftirqd/022:00:300
252419993219,4cyclictest131rcu_preempt20:55:190
252419993219,4cyclictest131rcu_preempt20:55:190
252419993217,4cyclictest131rcu_preempt23:25:130
252419993215,5cyclictest131rcu_preempt20:45:120
252419993214,5cyclictest131rcu_preempt22:15:150
252419993214,4cyclictest131rcu_preempt19:20:160
252419993214,4cyclictest131rcu_preempt19:20:160
252419993211,9cyclictest131rcu_preempt22:40:230
252419993211,9cyclictest131rcu_preempt22:40:230
252427993124,4cyclictest352101-21unixbench_singl23:55:252
252427993124,4cyclictest0-21swapper/223:25:262
252427993124,4cyclictest0-21swapper/223:20:242
252427993123,4cyclictest0-21swapper/222:45:142
252427993122,6cyclictest0-21swapper/220:55:012
252427993122,6cyclictest0-21swapper/220:55:002
25242499319,5cyclictest318060-21kworker/u8:3+events_freezable_power_22:25:021
252424993121,6cyclictest267882-21grep19:50:251
252424993121,6cyclictest267882-21grep19:50:241
25242499311,9cyclictest2776402ntp_kernel_pll_20:20:191
252424993119,7cyclictest0-21swapper/100:25:241
252424993116,9cyclictest131rcu_preempt19:15:171
252424993111,12cyclictest24-21ksoftirqd/122:25:141
252424993110,4cyclictest286051-21cut20:45:141
252419993120,4cyclictest131rcu_preempt21:20:170
252419993117,4cyclictest131rcu_preempt19:25:130
252419993117,4cyclictest131rcu_preempt19:25:120
252419993116,4cyclictest131rcu_preempt23:32:200
252427993022,4cyclictest0-21swapper/221:00:172
252424993022,4cyclictest0-21swapper/122:35:241
252424993021,6cyclictest308424-21fschecks_time21:50:171
252424993021,6cyclictest0-21swapper/122:40:261
252424993021,6cyclictest0-21swapper/122:40:251
252424993020,6cyclictest0-21swapper/121:25:021
252424993020,6cyclictest0-21swapper/120:30:261
252424993018,5cyclictest3346713-21kworker/u8:3+events_unbound19:25:251
252424993018,5cyclictest3346713-21kworker/u8:3+events_unbound19:25:241
252427992923,3cyclictest275515-21fschecks_count20:15:162
252427992921,5cyclictest315247-21fschecks_time22:10:162
252427992921,4cyclictest298813-21unixbench_singl21:20:262
252424992920,6cyclictest0-21swapper/122:15:271
252424992918,7cyclictest0-21swapper/119:30:151
252424992910,6cyclictest131rcu_preempt20:55:111
252424992910,6cyclictest131rcu_preempt20:55:101
25241999298,14cyclictest11-21ksoftirqd/000:20:570
25241999298,14cyclictest11-21ksoftirqd/000:20:560
252419992915,4cyclictest131rcu_preempt23:45:200
252427992819,6cyclictest0-21swapper/223:50:002
252427992819,6cyclictest0-21swapper/221:40:002
252424992815,7cyclictest131rcu_preempt20:38:161
252424992814,8cyclictest131rcu_preempt23:50:001
252424992814,8cyclictest131rcu_preempt00:10:171
252427992720,4cyclictest364901-21munin-plugin-st00:34:592
252427992720,4cyclictest356169-21munin-plugin-st00:10:012
252427992720,3cyclictest0-21swapper/220:00:002
252427992719,4cyclictest0-21swapper/223:30:262
252427992718,5cyclictest353825-21head00:00:252
25242799271,4cyclictest131rcu_preempt21:35:002
25242499271,7cyclictest284946-21head20:40:251
252427992619,4cyclictest0-21swapper/219:10:242
252427992615,8cyclictest358869-21ntp_states00:15:222
252427992613,10cyclictest252410-21cyclictest22:50:112
25242499268,12cyclictest24-21ksoftirqd/119:40:311
252427992518,4cyclictest345076-21head23:35:242
252427992518,4cyclictest278933-21grep20:25:162
252427992518,4cyclictest273435-21sh20:10:002
252427992418,3cyclictest0-21swapper/222:05:222
252427992418,3cyclictest0-21swapper/220:20:102
252427992414,8cyclictest295925-21wc21:15:002
252427992413,7cyclictest308368-21tune2fs21:50:152
252427992317,3cyclictest0-21swapper/220:30:252
252427992315,5cyclictest306064-21sh21:45:012
252427992314,5cyclictest285386-21auditctl20:45:002
252427992312,7cyclictest1-21systemd21:56:122
25242799231,20cyclictest311ktimers/200:25:212
252427992311,8cyclictest204-21dbus-daemon23:16:312
252427992216,3cyclictest0-21swapper/221:05:222
252427992216,3cyclictest0-21swapper/219:30:242
252427992215,5cyclictest318978-21ntp_kernel_err22:20:192
252427992215,5cyclictest264560-21kworker/u8:0+events_unbound19:53:192
252427992215,5cyclictest264560-21kworker/u8:0+events_unbound19:53:192
252427992214,5cyclictest0-21swapper/222:35:162
252427992214,5cyclictest0-21swapper/200:15:002
252427992213,6cyclictest0-21swapper/222:15:242
252427992116,2cyclictest0-21swapper/200:35:192
252427992115,4cyclictest326563-21kworker/u8:1+events_unbound23:12:492
252427992115,4cyclictest321527-21kworker/u8:1+events_unbound22:30:072
252427992115,3cyclictest287290-21sh20:50:002
252427992115,3cyclictest0-21swapper/223:50:242
252427992115,3cyclictest0-21swapper/223:00:152
252427992115,3cyclictest0-21swapper/221:15:252
252427992115,3cyclictest0-21swapper/220:10:262
25242799205,3cyclictest360026-21apt-get00:20:122
25242799205,3cyclictest360026-21apt-get00:20:112
252427992014,4cyclictest256802-21fschecks_time19:20:162
252427992014,4cyclictest256802-21fschecks_time19:20:152
252427992014,3cyclictest0-21swapper/219:45:172
252427992013,4cyclictest311ktimers/220:00:262
25242799197,3cyclictest131rcu_preempt22:25:172
252427991913,4cyclictest325845-21ntpq22:40:222
252427991913,4cyclictest325845-21ntpq22:40:212
252427991811,5cyclictest311ktimers/219:25:152
252427991811,5cyclictest311ktimers/219:25:152
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional