You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-26 - 13:38

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #4, slot #1

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack4slot1s.osadl.org (updated Wed Aug 26, 2026 00:44:12)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2258821290,3chrt0-21swapper/320:14:133
3057721100,2sleep30-21swapper/320:34:193
223552950,2chrt22351-21acpi20:14:111
70552910,2sleep00-21swapper/020:59:160
70552910,2sleep00-21swapper/020:59:160
317622910,3sleep30-21swapper/323:29:223
317622910,3sleep30-21swapper/323:29:223
181272830,3sleep118132-21sed00:19:251
181272830,3sleep118132-21sed00:19:251
123602790,3sleep3431rcuc/319:44:223
85312780,3sleep30-21swapper/321:04:023
85312780,3sleep30-21swapper/321:04:023
255242690,7sleep1271rcuc/121:49:061
255242690,7sleep1271rcuc/121:49:061
215072660,4sleep30-21swapper/320:09:223
298092650,5sleep129813-21timerwakeupswit23:24:221
210482600,2sleep121003-21latency_hist23:04:001
210482600,2sleep121003-21latency_hist23:04:001
32554995645,6cyclictest0-21swapper/000:04:030
32554995645,6cyclictest0-21swapper/000:04:030
239212560,3sleep123107-21/usr/sbin/munin23:09:181
239212560,3sleep123107-21/usr/sbin/munin23:09:181
87532520,2chrt0-21swapper/323:54:233
87532520,2chrt0-21swapper/323:54:233
304972520,3sleep130492-21gzip20:34:171
32554995039,6cyclictest0-21swapper/000:04:120
3255499501,6cyclictest16419-21taskset19:57:490
3255499501,6cyclictest16419-21taskset19:57:490
3255499490,32cyclictest17077-21runrttasks22:51:280
3255499471,7cyclictest10855-21chrt19:43:430
3255499471,7cyclictest10855-21chrt19:43:430
3255499471,6cyclictest1057-21latency_hist20:44:040
3255499471,6cyclictest1057-21latency_hist20:44:040
239022460,3sleep20-21swapper/220:17:582
3255499451,6cyclictest26939-21runrttasks21:50:460
3255499451,6cyclictest19689-21chrt21:33:370
3255499451,6cyclictest11487-21taskset22:37:010
3255499451,6cyclictest11487-21taskset22:37:010
47242440,3sleep20-21swapper/222:19:142
47242440,3sleep20-21swapper/222:19:142
3255499441,7cyclictest749-21runrttasks23:42:170
3255499441,7cyclictest749-21runrttasks23:42:170
3255499441,6cyclictest27727-21seq20:28:530
3255499440,6cyclictest25794-21chrt20:23:140
3255499440,6cyclictest25794-21chrt20:23:140
3255499431,6cyclictest9745-21taskset22:33:380
3255499431,6cyclictest9745-21taskset22:33:380
3255499430,7cyclictest2784-21runrttasks20:48:550
3255499430,7cyclictest2784-21runrttasks20:48:550
3255499420,9cyclictest18276-21taskset20:02:430
3255499420,9cyclictest18276-21taskset20:02:430
3255499420,8cyclictest28884-21runrttasks21:56:490
3255499420,8cyclictest28884-21runrttasks21:56:490
3255499420,7cyclictest22262-21taskset00:33:230
3255499420,7cyclictest22262-21taskset00:33:230
63342410,3sleep30-21swapper/323:49:163
3255499411,6cyclictest1464-21taskset23:36:260
3255499411,6cyclictest1464-21taskset23:36:260
3255499411,6cyclictest14448-21entropy21:19:130
3255499411,6cyclictest14448-21entropy21:19:130
253942410,2chrt0-21swapper/021:49:010
253942410,2chrt0-21swapper/021:49:010
151622410,2sleep1281ktimers/100:14:101
150462410,1sleep10-21swapper/100:14:061
3255499391,9cyclictest18968-21seq22:56:510
3255499391,9cyclictest18968-21seq22:56:510
3255499391,9cyclictest13034-21chrt21:14:160
3255499391,7cyclictest27197-21grep23:19:130
3255499391,7cyclictest27197-21grep23:19:130
32554993915,22cyclictest3333-21grep20:49:140
32554993915,22cyclictest3333-21grep20:49:140
32554993914,23cyclictest4689-21head22:19:140
32554993914,23cyclictest4689-21head22:19:140
3255499390,6cyclictest16427-21runrttasks00:15:340
3255699385,29cyclictest11634-21/usr/sbin/munin00:04:141
3255499381,8cyclictest14489-21ntpq22:44:170
3255499381,8cyclictest14489-21ntpq22:44:170
3255499381,7cyclictest30026-21ntpq21:59:170
3255499381,7cyclictest30026-21ntpq21:59:170
3255499381,7cyclictest13139-21fschecks_count19:49:120
32554993814,22cyclictest0-21swapper/021:09:220
3255499381,10cyclictest11233-21df19:44:110
161072380,2sleep30-21swapper/321:24:043
161072380,2sleep30-21swapper/321:24:043
3255499372,13cyclictest21996-21ntp_states23:04:180
3255499371,8cyclictest3642-21/usr/sbin/munin22:14:230
3255499371,8cyclictest23016-21needreboot20:14:170
3255499371,8cyclictest13220-21taskset22:40:220
3255499371,8cyclictest13220-21taskset22:40:220
3255499371,34cyclictest23678-21unixbench-2d00:34:240
3255499371,34cyclictest23678-21unixbench-2d00:34:240
32554993713,22cyclictest32553-21cyclictest23:18:590
3255499371,11cyclictest16327-21df21:24:120
3255499371,11cyclictest16327-21df21:24:120
251492370,2sleep125204-21wc00:39:201
251492370,2sleep125204-21wc00:39:201
205862370,3sleep30-21swapper/300:29:043
205862370,3sleep30-21swapper/300:29:043
136522370,2chrt0-21swapper/122:44:051
136522370,2chrt0-21swapper/122:44:051
3255699365,26cyclictest11254-21sed00:04:081
3255699365,26cyclictest11254-21sed00:04:081
3255499361,9cyclictest30580-21ntp_states20:34:190
3255499361,9cyclictest20595-21/usr/sbin/munin20:09:120
3255499361,8cyclictest29048-21cat20:29:370
3255499361,8cyclictest23363-21sshd21:42:580
3255499361,8cyclictest23363-21sshd21:42:580
3255499361,6cyclictest5887-21switchtime20:54:300
3255499361,6cyclictest5887-21switchtime20:54:300
3255499361,6cyclictest2310-21latency_hist22:14:020
3255499361,6cyclictest2310-21latency_hist22:14:020
32554993616,18cyclictest0-21swapper/019:34:210
3255499361,5cyclictest29475-21ntp_states23:24:190
32554993614,20cyclictest0-21swapper/023:49:230
32554993613,21cyclictest0-21swapper/022:59:130
32554993613,21cyclictest0-21swapper/022:59:130
86322350,3sleep20-21swapper/219:34:242
85812350,1sleep18597-21date21:04:051
85812350,1sleep18597-21date21:04:051
32556993530,4cyclictest6026-21basename22:23:591
32556993530,4cyclictest6026-21basename22:23:591
32556993527,6cyclictest312-21gltestperf23:34:151
32556993527,6cyclictest312-21gltestperf23:34:151
3255499351,8cyclictest8309-21runrttasks23:54:190
3255499351,8cyclictest8309-21runrttasks23:54:190
3255499351,5cyclictest25568-21unixbench_multi00:39:230
3255499351,5cyclictest25568-21unixbench_multi00:39:230
3255499351,4cyclictest31275-21df_inode22:04:120
3255499351,12cyclictest5032-21head23:44:220
3255499351,12cyclictest5032-21head23:44:220
3255499351,10cyclictest19519-21nfs4_client00:24:180
3255499351,10cyclictest19519-21nfs4_client00:24:180
3255499350,7cyclictest7984-21latency_hist22:29:020
3255499350,7cyclictest13551-21/usr/sbin/munin00:09:150
283212350,2sleep128433-21timerandwakeup21:54:221
283212350,2sleep128433-21timerandwakeup21:54:221
180482350,2sleep20-21swapper/222:54:162
180482350,2sleep20-21swapper/222:54:162
32556993428,5cyclictest14826-21kernelversion21:19:161
32556993428,5cyclictest14826-21kernelversion21:19:161
3255499341,9cyclictest17776-21rt-features00:19:230
3255499341,9cyclictest17776-21rt-features00:19:230
3255499341,8cyclictest9477-21ps21:04:190
3255499341,11cyclictest31177-21memory23:29:150
3255499341,11cyclictest31177-21memory23:29:150
3255499340,6cyclictest24821-21date23:14:010
3255499340,6cyclictest24821-21date23:14:010
3255499340,32cyclictest20055-21df21:34:110
3255499340,32cyclictest20055-21df21:34:110
32556993326,5cyclictest18051-21latency_hist21:29:061
32556993326,5cyclictest18051-21latency_hist21:29:061
32556993325,6cyclictest21445-21cut23:04:111
32556993324,7cyclictest6526-21cut22:24:101
3255499331,7cyclictest18863-21tune2fs20:04:120
284342330,3sleep20-21swapper/221:54:222
284342330,3sleep20-21swapper/221:54:222
103832330,1sleep20-21swapper/219:39:232
103832330,1sleep20-21swapper/219:39:232
32556993226,4cyclictest29399-21kworker/u9:4-xprtiod20:53:591
32556993226,4cyclictest29399-21kworker/u9:4-xprtiod20:53:591
32556993224,6cyclictest14028-21threads19:49:221
182672320,2chrt18269-21fschecks_count21:29:121
32556993126,3cyclictest0-21swapper/121:43:591
32556993126,3cyclictest0-21swapper/121:43:591
31282310,4sleep30-21swapper/322:14:173
32556992924,4cyclictest545-21in:imuxsock23:18:591
32556992924,3cyclictest32553-21cyclictest21:04:251
32556992923,4cyclictest0-21swapper/119:39:211
32556992923,4cyclictest0-21swapper/119:39:211
32556992921,6cyclictest23099-21sort00:34:161
32556992921,6cyclictest23099-21sort00:34:161
256662290,4sleep20-21swapper/221:49:112
32556992824,2cyclictest0-21swapper/120:33:591
32556992822,4cyclictest4109-21cron22:18:591
32556992822,4cyclictest0-21swapper/121:49:121
32556992722,4cyclictest1-21systemd23:40:591
32556992722,4cyclictest1-21systemd23:40:591
32556992722,3cyclictest101-21jbd2/mmcblk0p2-820:04:131
32556992720,5cyclictest32553-21cyclictest19:34:301
32556992621,4cyclictest6626-21latency_hist20:59:011
32556992621,4cyclictest6626-21latency_hist20:59:011
32556992621,3cyclictest0-21swapper/120:48:591
32556992621,3cyclictest0-21swapper/120:48:591
32556992620,4cyclictest21058-21idleruntime00:29:141
32556992620,4cyclictest21058-21idleruntime00:29:141
32556992619,5cyclictest14115-21latency_hist21:18:591
32556992618,6cyclictest32282-21ls22:04:231
149322260,2chrt14935-21unixbench-2d22:44:221
149322260,2chrt14935-21unixbench-2d22:44:221
32557992519,4cyclictest78712sleep223:54:132
32557992519,4cyclictest78712sleep223:54:132
32556992521,2cyclictest0-21swapper/123:34:001
32556992521,2cyclictest0-21swapper/123:34:001
32556992521,2cyclictest0-21swapper/119:54:151
32556992521,2cyclictest0-21swapper/119:54:151
32556992520,2cyclictest0-21swapper/123:53:591
32556992520,2cyclictest0-21swapper/123:48:591
32556992520,2cyclictest0-21swapper/123:48:591
32556992519,4cyclictest547-21rs:main1
32556992519,4cyclictest547-21rs:main1
32556992518,5cyclictest17682-21fschecks_count22:54:131
32556992518,5cyclictest17682-21fschecks_count22:54:131
32556992518,5cyclictest0-21swapper/120:39:131
32556992518,5cyclictest0-21swapper/120:39:131
32556992517,6cyclictest10934-21grep22:34:211
32556992517,6cyclictest10934-21grep22:34:211
32556992516,7cyclictest27030-21grep20:24:201
3255899241,16cyclictest10001-21taskset19:39:193
3255899241,16cyclictest10001-21taskset19:39:193
32556992418,4cyclictest27003-21fschecks_count23:19:121
32556992418,4cyclictest27003-21fschecks_count23:19:121
32556992418,4cyclictest0-21swapper/123:54:151
32556992418,4cyclictest0-21swapper/123:54:151
32556992418,4cyclictest0-21swapper/121:09:231
32556992418,4cyclictest0-21swapper/100:28:591
32556992418,4cyclictest0-21swapper/100:28:591
32556992416,5cyclictest18582-21latency_hist20:04:061
32556992416,5cyclictest18582-21latency_hist20:04:061
32556992415,7cyclictest15844-21fschecks_time22:49:111
3255899231,12cyclictest11174-21chrt00:04:033
3255899231,12cyclictest11174-21chrt00:04:033
32556992319,3cyclictest20496-21acpi20:09:111
32556992319,2cyclictest0-21swapper/121:34:221
32556992319,2cyclictest0-21swapper/121:34:221
32556992317,4cyclictest0-21swapper/119:44:121
32556992316,4cyclictest0-21swapper/122:09:211
32556992316,4cyclictest0-21swapper/122:09:211
131872230,3sleep30-21swapper/321:14:193
32556992115,3cyclictest0-21swapper/122:29:251
32556992115,3cyclictest0-21swapper/122:29:251
26512210,3sleep20-21swapper/223:39:182
26512210,3sleep20-21swapper/223:39:182
186552210,2sleep30-21swapper/300:24:013
186552210,2sleep30-21swapper/300:24:013
3255899203,11cyclictest0-21swapper/300:10:013
3255799202,12cyclictest0-21swapper/200:04:032
3255799202,12cyclictest0-21swapper/200:04:032
32556992016,2cyclictest0-21swapper/120:23:591
32556992016,2cyclictest0-21swapper/120:23:591
7642191,2sleep20-21swapper/222:09:142
7642191,2sleep20-21swapper/222:09:142
3255899193,9cyclictest0-21swapper/320:25:583
3255899193,9cyclictest0-21swapper/300:04:143
3255799194,9cyclictest0-21swapper/200:04:122
3255899183,9cyclictest0-21swapper/320:44:573
3255899183,9cyclictest0-21swapper/320:44:573
32557991813,3cyclictest93882sleep221:04:172
163262180,3chrt0-21swapper/321:24:123
163262180,3chrt0-21swapper/321:24:123
3255899173,9cyclictest0-21swapper/323:39:593
3255899173,9cyclictest0-21swapper/323:39:593
3255899172,10cyclictest0-21swapper/322:24:073
3255899172,10cyclictest0-21swapper/322:24:073
3255899171,11cyclictest13972-21taskset22:44:113
3255899171,11cyclictest13972-21taskset22:44:113
3255799171,13cyclictest4013-21chrt23:44:122
3255799171,13cyclictest4013-21chrt23:44:122
3255799171,12cyclictest0-21swapper/200:39:112
3255799171,12cyclictest0-21swapper/200:39:112
3255899168,5cyclictest241192sleep321:44:153
3255899168,5cyclictest241192sleep321:44:153
3255899163,9cyclictest0-21swapper/323:21:593
3255899163,9cyclictest0-21swapper/323:21:593
3255899163,9cyclictest0-21swapper/323:00:583
3255899163,9cyclictest0-21swapper/323:00:583
3255899163,9cyclictest0-21swapper/322:24:583
3255899163,9cyclictest0-21swapper/319:49:583
3255899161,9cyclictest0-21swapper/319:34:113
3255899161,11cyclictest8674-21taskset21:04:103
3255899161,11cyclictest25561-21chrt23:14:143
3255899161,11cyclictest18828-21chrt21:29:183
3255899161,11cyclictest17651-21chrt22:54:123
3255899161,11cyclictest17651-21chrt22:54:123
3255899161,11cyclictest0-21swapper/300:29:143
3255899161,11cyclictest0-21swapper/300:29:143
3255799169,4cyclictest151892sleep200:14:122
3255799163,9cyclictest0-21swapper/222:35:182
3255799163,9cyclictest0-21swapper/222:35:182
3255799163,9cyclictest0-21swapper/220:09:582
32557991611,3cyclictest170202sleep219:59:122
32557991611,3cyclictest170202sleep219:59:122
3255799161,12cyclictest32496-21taskset20:39:202
3255799161,12cyclictest32496-21taskset20:39:202
3255799161,12cyclictest20739-21taskset00:29:112
3255799161,12cyclictest20739-21taskset00:29:112
3255799161,12cyclictest20416-21chrt22:59:222
3255799161,12cyclictest20416-21chrt22:59:222
3255799161,11cyclictest18787-21chrt00:24:082
3255799161,11cyclictest18787-21chrt00:24:082
3255799161,11cyclictest16099-21taskset22:49:132
3255899153,9cyclictest0-21swapper/319:54:583
3255899153,9cyclictest0-21swapper/319:54:583
3255899152,9cyclictest0-21swapper/323:45:583
3255899152,9cyclictest0-21swapper/323:45:583
3255899151,9cyclictest17798-21chrt19:59:213
3255899151,9cyclictest17798-21chrt19:59:213
3255899151,9cyclictest0-21swapper/323:09:093
3255899151,9cyclictest0-21swapper/323:09:093
3255899151,10cyclictest8865-21taskset22:29:203
3255899151,10cyclictest8865-21taskset22:29:203
3255899151,10cyclictest32512-21chrt23:34:123
3255899151,10cyclictest32512-21chrt23:34:123
3255899151,10cyclictest29765-21chrt21:59:133
3255899151,10cyclictest29765-21chrt21:59:133
3255899151,10cyclictest29368-21chrt23:24:153
3255899151,10cyclictest28162-21taskset21:54:193
3255899151,10cyclictest28162-21taskset21:54:193
3255899151,10cyclictest21990-21chrt23:04:183
3255899151,10cyclictest12339-21chrt22:39:133
3255899151,10cyclictest12339-21chrt22:39:133
3255899151,10cyclictest0-21swapper/300:14:123
3255799159,4cyclictest12542sleep220:44:122
3255799159,4cyclictest12542sleep220:44:122
3255799153,8cyclictest0-21swapper/220:55:592
3255799153,8cyclictest0-21swapper/220:55:592
3255799152,11cyclictest0-21swapper/221:24:112
3255799152,11cyclictest0-21swapper/221:24:112
3255799152,10cyclictest0-21swapper/221:39:152
3255799152,10cyclictest0-21swapper/221:39:152
3255799151,9cyclictest0-21swapper/223:09:092
3255799151,9cyclictest0-21swapper/223:09:092
3255799151,9cyclictest0-21swapper/221:49:032
3255799151,9cyclictest0-21swapper/221:49:032
3255799151,12cyclictest0-21swapper/219:54:102
3255799151,12cyclictest0-21swapper/219:54:102
3255799151,11cyclictest3191-21chrt20:49:132
3255799151,11cyclictest3191-21chrt20:49:132
3255799151,11cyclictest26476-21chrt20:24:132
3255799151,11cyclictest19214-21chrt20:04:152
3255799151,11cyclictest15542-21chrt21:19:242
3255799151,11cyclictest15542-21chrt21:19:242
3255799151,10cyclictest417-21chrt23:34:152
3255799151,10cyclictest417-21chrt23:34:152
3255799151,10cyclictest27077-21chrt23:19:122
3255799151,10cyclictest27077-21chrt23:19:122
3255799151,10cyclictest0-21swapper/222:49:052
3255799151,10cyclictest0-21swapper/222:49:052
227192150,2sleep20-21swapper/200:34:132
227192150,2sleep20-21swapper/200:34:132
3255899149,3cyclictest64212chrt20:57:263
3255899149,3cyclictest64212chrt20:57:263
3255899149,3cyclictest325142chrt22:05:163
3255899149,3cyclictest121352chrt21:13:303
3255899148,3cyclictest0-21swapper/322:37:083
3255899148,3cyclictest0-21swapper/322:37:083
3255899142,9cyclictest0-21swapper/321:44:043
3255899142,9cyclictest0-21swapper/321:44:043
3255899141,9cyclictest24961-21chrt20:19:193
3255899141,9cyclictest24961-21chrt20:19:193
3255899141,8cyclictest0-21swapper/320:29:103
3255899141,7cyclictest0-21swapper/321:34:213
3255899141,7cyclictest0-21swapper/321:34:213
3255799149,3cyclictest37702chrt22:14:402
3255799141,11cyclictest32227-21chrt22:04:232
3255799141,11cyclictest0-21swapper/222:29:142
3255799141,11cyclictest0-21swapper/222:29:142
3255799141,11cyclictest0-21swapper/220:34:012
3255799141,10cyclictest8506-21chrt21:04:012
3255799141,10cyclictest8506-21chrt21:04:012
3255799141,10cyclictest21622-21chrt23:04:122
3255899138,3cyclictest269072chrt21:50:213
3255899138,3cyclictest241062chrt00:38:083
3255899138,3cyclictest241062chrt00:38:083
3255899138,3cyclictest20782chrt22:11:513
3255899138,3cyclictest20782chrt22:11:513
3255899131,9cyclictest514-21chrt20:39:253
3255899131,9cyclictest514-21chrt20:39:253
3255899131,9cyclictest16388-21taskset22:49:183
3255899131,8cyclictest3391-21chrt20:49:143
3255899131,8cyclictest3391-21chrt20:49:143
3255899131,8cyclictest0-21swapper/320:04:153
3255799138,3cyclictest315972sleep223:29:212
3255799138,3cyclictest315972sleep223:29:212
3255799138,3cyclictest212722chrt21:35:002
3255799138,3cyclictest212722chrt21:35:002
3255799138,3cyclictest124922chrt19:45:302
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional