You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-07 - 09:15

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Phytec
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot5s.osadl.org (updated Thu May 07, 2026 00:44:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3114394995647,5cyclictest3171629-21cron21:55:012
3114394995119,29cyclictest3157773-21sh21:15:002
3114394995029,17cyclictest0-21swapper/220:40:022
3114390994937,9cyclictest231ktimers/121:00:141
3114390994720,23cyclictest0-21swapper/123:25:271
3114385994734,10cyclictest121ktimers/023:05:130
3114399994537,6cyclictest3160873-21kworker/u8:0+events_unbound22:21:053
3114399994436,5cyclictest3125153-21munin-plugin-st19:40:013
3114399994436,5cyclictest3125153-21munin-plugin-st19:40:003
3114394994319,20cyclictest0-21swapper/200:30:202
311439499431,11cyclictest3154362-21cron21:05:002
3114390994335,5cyclictest3183799-21munin-plugin-st22:30:001
3114390994333,7cyclictest0-21swapper/119:35:011
3114390994333,7cyclictest0-21swapper/119:35:011
3114390994314,26cyclictest3206557-21fschecks_count23:35:151
3114399994233,5cyclictest0-21swapper/322:30:263
311439099421,9cyclictest3175960-21/usr/sbin/munin22:05:211
311439099421,10cyclictest3151813-21/usr/sbin/munin20:55:221
3114385994231,8cyclictest121ktimers/023:55:130
3114385994228,10cyclictest121ktimers/020:50:130
3114399994130,6cyclictest0-21swapper/319:55:383
311439499419,6cyclictest3146298-21cut20:40:152
311439099413,35cyclictest3217734-21/usr/sbin/munin00:05:201
311439099411,9cyclictest3170991-21sed21:50:231
311439099411,8cyclictest3210141-21/usr/sbin/munin23:45:131
3114390994112,26cyclictest3168878-21ls21:45:181
3114385994132,7cyclictest121ktimers/019:20:120
3114399994030,7cyclictest391ktimers/323:20:113
311439499409,7cyclictest3217913-21wc00:05:232
3114394994016,19cyclictest0-21swapper/221:35:002
3114394994012,3cyclictest3141163-21fschecks_time20:25:162
311439499401,10cyclictest3208809-21find23:40:232
311439099401,8cyclictest3144468-21/usr/sbin/munin20:35:141
3114390994016,21cyclictest0-21swapper/119:25:271
3114390994010,27cyclictest204-21dbus-daemon22:11:021
3114390994010,27cyclictest204-21dbus-daemon22:11:021
3114385994033,4cyclictest3227217-21latency_hist00:35:000
311439499399,4cyclictest3223201-21unixbench_multi00:20:252
311439499397,9cyclictest3165378-21/usr/sbin/munin21:35:202
311439499397,7cyclictest1-21systemd22:21:032
311439499393,7cyclictest3161196-21cron21:25:002
311439499393,7cyclictest3161196-21cron21:25:002
3114394993931,4cyclictest3150947-21debian-sa120:55:002
3114394993929,7cyclictest3177985-21grep22:10:252
3114394993929,7cyclictest3177985-21grep22:10:242
311439499391,8cyclictest3161648-21cut21:25:162
311439499391,8cyclictest3161648-21cut21:25:152
311439499391,11cyclictest3116653-21fschecks_count19:15:042
311439499391,10cyclictest3188588-21sshd22:41:082
311439099399,28cyclictest3139933-21grep20:20:221
311439099399,27cyclictest3199752-21cut23:15:161
311439099391,9cyclictest3181091-21/usr/sbin/munin22:20:221
311439099391,8cyclictest3134676-21/usr/sbin/munin20:05:211
311439099391,8cyclictest3134676-21/usr/sbin/munin20:05:211
311439099391,8cyclictest3125607-21/usr/sbin/munin19:40:111
311439099391,7cyclictest3149663-21/usr/sbin/munin20:50:161
311439099391,6cyclictest3148858-21users20:45:281
3114399993828,6cyclictest3186729-21unixbench_multi22:35:243
311439499388,7cyclictest3119464-21cat19:20:242
3114394993831,4cyclictest0-21swapper/223:10:232
311439499381,9cyclictest3186757-21head22:35:242
311439499381,9cyclictest3169930-21cron21:50:012
311439499381,9cyclictest3114686-21grep19:10:152
3114394993815,3cyclictest3194475-21/usr/sbin/munin23:00:252
3114394993813,3cyclictest0-21swapper/220:05:422
3114394993813,3cyclictest0-21swapper/220:05:412
311439499381,11cyclictest3121339-21head19:25:282
311439499381,10cyclictest3131733-21unixbench_multi19:55:252
311439099381,9cyclictest3198721-21timerandwakeup23:10:211
3114390993819,15cyclictest0-21swapper/120:40:141
311439099381,8cyclictest3184613-21/usr/sbin/munin22:30:221
311439099381,8cyclictest3155169-21/usr/sbin/munin21:05:221
311439099381,7cyclictest3208795-21/usr/sbin/munin23:40:221
311439099381,7cyclictest3164169-21awk21:30:241
311439099381,7cyclictest3123800-21/usr/sbin/munin19:35:151
311439099381,7cyclictest3123800-21/usr/sbin/munin19:35:151
311439099381,7cyclictest3114664-21/usr/sbin/munin19:10:151
311439099381,6cyclictest3228116-21/usr/sbin/munin00:35:231
311439099381,6cyclictest3194589-21/usr/sbin/munin23:00:141
311439099381,5cyclictest3172318-21meminfo21:55:211
311439099381,4cyclictest3158122-21/usr/sbin/munin21:15:141
3114385993830,4cyclictest3219823-21grep00:10:240
3114385993829,5cyclictest0-21swapper/021:20:010
311439499379,5cyclictest0-21swapper/200:00:012
311439499371,8cyclictest3207462-21head23:35:242
3114394993712,3cyclictest0-21swapper/220:10:152
311439499371,14cyclictest3125151-21sh19:40:002
311439499371,14cyclictest3125151-21sh19:40:002
311439499371,11cyclictest3151628-21meminfo20:55:182
311439499371,10cyclictest3193815-21grep22:55:262
311439099373,6cyclictest3225955-21irqstats00:30:171
311439099371,9cyclictest3196297-21df_inode23:05:151
311439099371,9cyclictest3136396-21ntp_states20:10:201
311439099371,9cyclictest3130080-21users19:50:241
311439099371,8cyclictest3201401-21/usr/sbin/munin23:20:141
311439099371,7cyclictest3138126-21/usr/sbin/munin20:15:241
311439099371,6cyclictest3214000-21ntp_states23:55:211
311439099371,6cyclictest3142790-21/usr/sbin/munin20:30:151
311439099371,6cyclictest3132664-21/usr/sbin/munin20:00:151
311439099371,5cyclictest3165316-21/usr/sbin/munin21:35:191
311439099371,5cyclictest3118848-21/usr/sbin/munin19:20:161
3114390993715,19cyclictest0-21swapper/122:40:181
311439099371,4cyclictest3221250-21/usr/sbin/munin00:15:241
311439099371,4cyclictest3221250-21/usr/sbin/munin00:15:231
311439099371,4cyclictest3215365-21/usr/sbin/munin00:00:261
311439099371,4cyclictest3173595-21/usr/sbin/munin22:00:141
311439099371,4cyclictest3131827-21sshd19:55:491
311439099371,3cyclictest3222314-21/usr/sbin/munin00:20:141
311439099371,3cyclictest3189279-21/usr/sbin/munin22:45:161
311439099371,3cyclictest3156424-21/usr/sbin/munin21:10:151
311439099371,3cyclictest3117036-21/usr/sbin/munin19:15:151
311439099371,10cyclictest3211737-21dpkg23:50:121
3114385993730,4cyclictest3139014-21sh20:20:000
3114385993727,7cyclictest3175053-21sh22:05:010
3114385993725,7cyclictest0-21swapper/022:11:020
3114385993725,7cyclictest0-21swapper/022:11:020
311439499367,5cyclictest3202513-21seq23:21:572
311439499361,8cyclictest3211092-21head23:45:252
3114394993613,4cyclictest3211898-21fschecks_count23:50:152
3114394993612,5cyclictest311ktimers/222:45:122
311439499361,12cyclictest3133891-21cron20:05:002
3114394993611,2cyclictest0-21swapper/221:07:282
311439499361,10cyclictest3183783-21cron22:30:002
311439099362,9cyclictest3223902-21apt-get00:25:121
311439099361,6cyclictest3206177-21latency_hist23:35:001
311439099361,6cyclictest3185845-21diskstats22:35:141
311439099361,4cyclictest3219212-21meminfo00:10:181
311439099361,4cyclictest3191284-21/usr/sbin/munin22:50:181
311439099361,3cyclictest3161547-21/usr/sbin/munin21:25:151
311439099361,3cyclictest3161547-21/usr/sbin/munin21:25:141
311439099361,11cyclictest3166782-21grep21:40:151
3114390993610,22cyclictest3161307-21latency_hist21:25:001
3114390993610,22cyclictest3161307-21latency_hist21:25:001
3114385993625,6cyclictest3145836-21grep20:40:010
3114399993526,6cyclictest3115606-21grep19:10:253
311439499358,5cyclictest0-21swapper/220:30:222
3114394993514,17cyclictest0-21swapper/220:20:242
3114394993510,4cyclictest311ktimers/221:43:262
311439099352,4cyclictest3179410-21/usr/sbin/munin22:15:231
311439099351,7cyclictest3193597-21/usr/sbin/munin22:55:251
311439099351,6cyclictest3127873-21ntp_states19:45:211
3114385993524,7cyclictest0-21swapper/021:45:250
3114385993510,20cyclictest3125376-21latency_hist19:40:020
3114385993510,20cyclictest3125376-21latency_hist19:40:020
3114399993424,7cyclictest3195519-21unixbench_singl23:00:263
3114399993423,8cyclictest3150950-21audit_send_reply20:55:003
311439499342,10cyclictest3174588-21unixbench_multi22:00:262
311439499341,10cyclictest3173365-21basename22:00:002
311439099349,21cyclictest3141412-21cat20:25:191
3114385993426,4cyclictest0-21swapper/023:16:160
3114385993423,7cyclictest3171231-21grep21:50:250
311439999333,5cyclictest131rcu_preempt19:20:123
3114399993327,4cyclictest3167986-21kworker/u8:3+events_unbound00:03:223
3114399993327,4cyclictest3154146-21kworker/u8:2+events_unbound21:15:543
3114399993324,5cyclictest3172856-21tail21:55:263
311439499331,12cyclictest3175626-21irqstats22:05:162
311439499331,11cyclictest3203190-21grep23:25:172
311439499331,11cyclictest3137653-21diskstats20:15:152
311439499331,10cyclictest3128881-21wc19:50:022
3114394993310,3cyclictest3185498-21latency_hist22:35:012
3114394993310,3cyclictest0-21swapper/200:29:592
311438599330,3cyclictest0-21swapper/000:20:120
3114399993226,3cyclictest3225390-21kthreadd00:29:593
3114399993224,4cyclictest0-21swapper/300:10:243
3114399993223,4cyclictest0-21swapper/322:05:153
311439499329,5cyclictest0-21swapper/221:15:162
311439499322,11cyclictest3125841-21perl19:40:172
3114394993212,3cyclictest0-21swapper/219:35:012
3114394993212,3cyclictest0-21swapper/219:35:012
3114385993224,4cyclictest0-21swapper/023:35:150
3114385993224,4cyclictest0-21swapper/019:45:250
3114385993222,6cyclictest0-21swapper/023:50:250
311439999314,4cyclictest131rcu_preempt00:05:113
311439999313,5cyclictest131rcu_preempt20:40:113
3114399993125,4cyclictest3138876-21kworker/u8:3+events_unbound20:30:453
3114399993123,4cyclictest0-21swapper/320:05:273
3114399993123,4cyclictest0-21swapper/320:05:263
3114399993119,8cyclictest0-21swapper/321:45:023
311439499318,5cyclictest311ktimers/200:00:232
3114385993123,4cyclictest0-21swapper/022:50:240
3114385993123,3cyclictest0-21swapper/021:05:260
3114385993122,6cyclictest3183322-21seq22:25:250
3114385993122,6cyclictest121ktimers/000:00:070
3114399993024,4cyclictest3167986-21kworker/u8:3+events_unbound22:26:053
3114399993022,4cyclictest0-21swapper/322:50:243
3114399993019,8cyclictest3228867-21latency_hist00:40:013
3114399993019,8cyclictest3190197-21unixbench_singl22:45:243
3114399993010,4cyclictest131rcu_preempt20:00:113
311439499307,7cyclictest3222036-21ls00:20:002
311439499307,7cyclictest3222036-21ls00:20:002
311439499301,11cyclictest3130017-21sed19:50:222
3114385993022,4cyclictest0-21swapper/023:05:010
3114385993021,6cyclictest0-21swapper/022:00:000
3114385993020,7cyclictest3131756-21grep19:55:250
3114399992921,5cyclictest3148932-21kworker/u8:0+events_unbound21:09:003
3114399992921,4cyclictest0-21swapper/323:15:163
3114399992919,7cyclictest3152649-21sh21:00:013
3114385992921,4cyclictest0-21swapper/023:30:260
3114385992918,7cyclictest3211052-21grep23:45:250
3114385992917,8cyclictest3136825-21grep20:10:240
311438599290,3cyclictest0-21swapper/020:55:140
3114399992822,4cyclictest3123861-21kworker/u8:2+events_unbound20:20:433
3114399992821,5cyclictest3167986-21kworker/u8:3+events_unbound23:31:203
3114399992820,6cyclictest3211269-21kworker/u8:0+events_unbound23:50:243
3114399992819,5cyclictest3166313-21grep21:40:003
311439999281,3cyclictest131rcu_preempt20:25:123
311439499288,4cyclictest311ktimers/200:15:012
3114399992721,4cyclictest3167986-21kworker/u8:3+events_unbound23:58:273
3114399992720,5cyclictest3138876-21kworker/u8:3+events_unbound21:00:503
3114399992720,4cyclictest3148932-21kworker/u8:0+events_unbound21:12:583
3114399992720,3cyclictest0-21swapper/300:15:153
3114399992720,3cyclictest0-21swapper/300:15:153
3114399992719,5cyclictest3143988-21kworker/u8:1+events_unbound20:40:013
3114399992719,4cyclictest0-21swapper/321:25:153
3114399992719,4cyclictest0-21swapper/321:25:153
3114399992717,7cyclictest3198548-21seq23:10:203
3114385992721,3cyclictest0-21swapper/021:15:000
3114385992721,3cyclictest0-21swapper/000:15:190
3114385992721,3cyclictest0-21swapper/000:15:180
3114385992720,4cyclictest0-21swapper/019:15:010
3114385992719,5cyclictest0-21swapper/021:30:230
3114385992719,4cyclictest13550irq/42-30be0000.ethernet19:40:250
311439999263,4cyclictest0-21swapper/323:05:133
3114399992620,4cyclictest3211269-21kworker/u8:0+events_unbound00:30:153
3114399992620,3cyclictest0-21swapper/322:10:223
3114399992620,3cyclictest0-21swapper/322:10:213
3114385992618,5cyclictest0-21swapper/021:20:260
3114385992618,5cyclictest0-21swapper/021:20:250
3114385992617,6cyclictest0-21swapper/022:15:200
3114385992616,6cyclictest3186769-21grep22:35:240
3114399992519,4cyclictest3114376-21cyclictest21:45:223
3114399992519,3cyclictest0-21swapper/320:15:233
3114399992519,3cyclictest0-21swapper/319:15:253
3114399992518,4cyclictest0-21swapper/321:20:253
3114399992518,4cyclictest0-21swapper/321:20:243
3114399992516,5cyclictest0-21swapper/323:30:003
3114399992515,7cyclictest3208685-21ntp_states23:40:203
3114385992518,3cyclictest0-21swapper/020:20:220
3114385992517,4cyclictest0-21swapper/022:30:260
3114385992515,6cyclictest0-21swapper/023:25:260
311438599251,4cyclictest3116855-21apt-get19:15:130
3114399992418,3cyclictest0-21swapper/319:50:153
3114399992417,5cyclictest2012120-21kworker/u8:3+events_unbound19:26:283
3114399992417,4cyclictest0-21swapper/321:30:233
3114399992416,5cyclictest3148769-21tail20:45:263
3114399992415,6cyclictest0-21swapper/319:45:153
3114394992417,4cyclictest3201059-21cron23:20:012
3114394992414,6cyclictest0-21swapper/223:30:252
3114385992417,4cyclictest3228790-21sh00:40:000
3114385992417,4cyclictest3121746-21sh19:30:000
3114385992417,3cyclictest0-21swapper/020:30:240
3114385992416,6cyclictest121ktimers/019:30:120
3114385992416,6cyclictest121ktimers/019:30:120
3114385992416,5cyclictest0-21swapper/020:10:000
3114385992416,5cyclictest0-21swapper/020:10:000
3114385992416,4cyclictest3208257-21fschecks_count23:40:150
3114385992416,3cyclictest0-21swapper/000:05:140
311438599241,4cyclictest3153007-21df21:00:150
311438599241,19cyclictest0-21swapper/021:40:410
311438599241,19cyclictest0-21swapper/020:41:040
311438599241,19cyclictest0-21swapper/020:03:170
311438599241,19cyclictest0-21swapper/019:53:590
311438599240,20cyclictest0-21swapper/022:41:030
311439999239,10cyclictest391ktimers/321:50:113
3114399992316,3cyclictest0-21swapper/319:45:003
3114399992314,6cyclictest3222361-21fschecks_count00:20:173
3114385992318,3cyclictest13550irq/42-30be0000.ethernet21:40:010
311438599231,6cyclictest3190608-21cron22:50:000
3114385992316,3cyclictest0-21swapper/022:20:550
3114385992314,6cyclictest0-21swapper/022:05:200
3114385992312,7cyclictest3152305-1kworker/0:0H+mmc_complete00:30:000
311438599230,19cyclictest0-21swapper/020:27:250
3114399992216,4cyclictest3114376-21cyclictest22:40:233
3114399992216,3cyclictest0-21swapper/323:35:143
3114399992215,4cyclictest391ktimers/322:59:383
3114399992213,5cyclictest0-21swapper/319:30:263
3114399992213,5cyclictest0-21swapper/319:30:263
3114394992215,4cyclictest0-21swapper/223:05:162
3114394992215,3cyclictest0-21swapper/200:40:002
3114385992216,3cyclictest13550irq/42-30be0000.ethernet23:25:000
3114385992215,3cyclictest0-21swapper/021:27:000
3114385992215,3cyclictest0-21swapper/021:27:000
311438599221,19cyclictest0-21swapper/020:47:210
311438599220,18cyclictest0-21swapper/022:55:320
311439999217,5cyclictest3176646-21kworker/u8:2+events_unbound22:18:303
3114399992110,8cyclictest164-21systemd-journal20:12:403
3114394992114,4cyclictest0-21swapper/222:15:252
3114394992113,4cyclictest311ktimers/220:45:262
3114394992112,6cyclictest0-21swapper/222:55:002
3114399992014,4cyclictest3209304-21kworker/u8:1+events_unbound23:48:423
311438599201,5cyclictest3198325-21meminfo23:10:160
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional