You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-22 - 06:22

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #a, slot #8

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackaslot8s.osadl.org (updated Wed Jul 22, 2026 00:45:03)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
234952226104,100sleep30-21swapper/319:08:493
23380217398,24sleep20-21swapper/219:07:202
234562158103,28sleep10-21swapper/119:08:191
233852140103,26sleep00-21swapper/019:07:220
74402770,1chrt7304-21/usr/sbin/munin23:55:130
25052720,3sleep10-21swapper/123:02:571
254802700,2sleep00-21swapper/022:31:500
231602650,7sleep012-21ksoftirqd/023:18:290
278312610,1sleep00-21swapper/000:34:580
144872600,2sleep00-21swapper/023:11:310
116562600,2sleep10-21swapper/122:45:231
34382580,5sleep20-21swapper/222:15:232
255322550,2chrt0-21swapper/223:20:162
113732550,1sleep20-21swapper/223:10:002
282462540,1sleep30-21swapper/322:10:143
100462530,2sleep00-21swapper/021:32:230
43672520,3chrt11349-21diskmemload22:40:181
270252520,1sleep00-21swapper/022:56:460
271612510,1sleep30-21swapper/300:10:013
175332510,1sleep30-21swapper/300:26:183
75502490,2sleep30-21swapper/322:19:053
315282490,2sleep00-21swapper/021:24:560
23858994913,18cyclictest12694-21sh21:58:440
225652490,2sleep10-21swapper/100:05:331
23858994810,22cyclictest19817-21seq20:13:300
2385899479,24cyclictest13093-21seq21:35:120
2385899479,24cyclictest13093-21seq21:35:120
222612460,1sleep10-21swapper/122:30:051
153662460,1sleep10-21swapper/123:12:241
35922450,4sleep10-21swapper/123:27:411
30392440,1sleep00-21swapper/022:40:010
2385899447,23cyclictest17411-21sh23:38:060
187242440,1sleep20-21swapper/220:10:182
2385899436,20cyclictest7928-21sh23:06:400
23858994314,14cyclictest8992-21sh22:44:060
2385899411,5cyclictest18082-21rm21:14:590
2385899395,20cyclictest21254-21sh22:52:420
2385899389,16cyclictest11349-21diskmemload21:18:150
23858993511,21cyclictest20288-21cron00:05:000
2385899350,7cyclictest7058-21grep23:30:180
2385899340,3cyclictest0-21swapper/000:07:100
2385899340,2cyclictest3871-21ssh21:51:380
2385899330,4cyclictest29841-21ssh23:23:260
2385899330,3cyclictest9533-21ssh00:20:220
2385899330,3cyclictest32280-21ssh22:13:050
2385899330,3cyclictest18325-21ssh00:27:060
2385899323,24cyclictest32044-21rm23:25:120
2385899322,24cyclictest23901-21apt-get19:10:110
2385899322,24cyclictest23901-21apt-get19:10:110
23858993210,19cyclictest28839-21tune2fs19:20:150
23858993210,19cyclictest20186-21find20:15:130
2385899320,3cyclictest29420-21ssh21:46:300
2385899320,3cyclictest23826-21ssh22:06:220
2385899320,3cyclictest0-21swapper/020:20:140
2385899320,2cyclictest0-21swapper/020:45:180
219102320,2chrt0-21swapper/323:40:513
2385899316,22cyclictest830-21cat19:30:150
23858993114,14cyclictest2714-21sh23:50:440
23858993114,14cyclictest18573-21sh22:02:500
23858993113,15cyclictest12119-21sh22:21:530
2385899310,8cyclictest3620-21ssh21:27:420
2385899310,3cyclictest11349-21diskmemload00:18:460
2385899310,2cyclictest0-21swapper/022:45:500
323452300,2sleep10-21swapper/123:49:401
2385899308,20cyclictest19893-21expr23:40:130
2385899308,19cyclictest31488-21latency19:25:190
2385899301,4cyclictest27379-21apt-get00:10:120
23858993014,14cyclictest880-21sh23:01:180
23858993014,13cyclictest19720-21sh21:40:040
2385899300,6cyclictest14789-21proc_pri20:00:220
2385899300,6cyclictest14789-21proc_pri20:00:220
2385899300,5cyclictest10534-21gunzip21:05:260
2385899300,3cyclictest3319-21cstates20:50:140
2385899300,3cyclictest30592-21runrttasks23:47:540
2385899300,2cyclictest260642sleep020:25:580
2385899300,2cyclictest0-21swapper/020:55:180
2385899300,2cyclictest0-21swapper/020:05:260
2385899294,22cyclictest29803-21cat20:35:170
23858992913,14cyclictest32087-21sh00:37:310
23858992910,16cyclictest17636-21sh22:25:410
2385899290,4cyclictest26816-21cstates20:30:140
2385899290,3cyclictest12185-21hddtemp_smartct19:55:180
2386299284,16cyclictest8935-21sh22:20:123
2385999281,25cyclictest0-21swapper/119:16:251
2385899289,17cyclictest427-21seq20:43:110
2385899287,19cyclictest2850-21idleruntime22:15:170
2385899286,19cyclictest27827-21seq19:16:070
2385899282,2cyclictest0-21swapper/021:00:130
2385899281,24cyclictest11170-21apt19:54:590
2385899273,18cyclictest2383-21apt-get19:35:040
2386299261,24cyclictest0-21swapper/319:14:563
2386299261,24cyclictest0-21swapper/319:14:563
2386299261,23cyclictest0-21swapper/319:15:073
23860992619,5cyclictest27460-21/usr/sbin/munin19:15:262
2385899261,7cyclictest8625-21sed19:45:270
2385899261,4cyclictest5106-21tr19:40:130
2386299253,15cyclictest11349-21diskmemload21:30:133
2386299252,15cyclictest640-21snmpd22:45:123
23859992517,6cyclictest2680-21sh21:50:261
23860992417,5cyclictest386-21sh23:00:512
232502240,1sleep20-21swapper/222:05:472
2386299231,15cyclictest640-21snmpd20:45:123
2385999238,3cyclictest321152cat19:25:261
23859992315,5cyclictest6329-21sh23:30:121
23859992314,7cyclictest4002-21python319:35:241
52832220,7sleep3331rcuc/300:17:233
2386299222,14cyclictest24001-21ssh22:55:133
23862992212,7cyclictest28649-21cron22:35:003
23860992215,5cyclictest52950irq/141-enp3s0-21:25:002
23860992214,6cyclictest22752-21df22:30:162
23860992213,7cyclictest5176-21unixbench-2d22:40:252
23860992213,7cyclictest14936-21tune2fs22:00:162
2386099220,8cyclictest194572basename21:40:002
2386099220,8cyclictest194572basename21:40:002
23859992214,6cyclictest10025-21ls19:50:161
2386299211,5cyclictest0-21swapper/321:45:413
23862992114,5cyclictest5924-21ntpq19:40:213
23862992113,6cyclictest4907-21ls20:55:003
23862992111,8cyclictest19969-21sed23:15:273
2386299211,16cyclictest52750irq/139-enp3s0-22:05:153
23862992110,3cyclictest149872sh23:35:393
23860992113,6cyclictest560-21dbus-daemon00:40:002
23860992113,6cyclictest5354-21awk19:40:152
2386099211,10cyclictest3052-21cat00:15:252
2385999219,8cyclictest5004-21apt-get19:40:121
2385999212,7cyclictest4963-21apt-get23:05:121
23859992119,1cyclictest0-21swapper/119:14:551
23859992119,1cyclictest0-21swapper/119:14:541
2386299208,7cyclictest6005-21grep23:30:013
23862992014,4cyclictest32436-21timerandwakeup20:40:283
23862992012,6cyclictest7958-21sed19:45:203
23862992012,6cyclictest24493-21find20:25:133
23862992012,6cyclictest19028-21tune2fs21:15:153
23862992012,6cyclictest17958-21pluginstate23:15:003
2386299201,14cyclictest640-21snmpd23:50:163
2386099208,9cyclictest560-21dbus-daemon00:20:222
2386099203,10cyclictest27916-21ssh22:10:132
23860992014,4cyclictest13232-21sh22:46:372
23860992014,3cyclictest18789-21sh23:15:162
23860992013,4cyclictest31163-21apt-get20:40:152
23860992013,4cyclictest25929-21sh21:44:452
23860992012,6cyclictest5915-21memory20:55:202
23860992012,6cyclictest1427-21tune2fs21:50:142
23860992012,6cyclictest10148-21expr21:05:212
23860992011,7cyclictest2956-21cat20:50:002
2386099200,7cyclictest659-21ssh22:37:432
2385999208,5cyclictest26821-21cat21:45:141
23859992012,6cyclictest9372-21awk00:20:201
23859992012,6cyclictest30284-21seq19:24:541
23859992011,7cyclictest24913-21cat22:55:201
23859992011,6cyclictest27774-21which20:30:231
212962200,2sleep10-21swapper/122:52:461
6802190,1sleep10-21swapper/119:30:131
320082190,2sleep30-21swapper/300:13:223
2386299198,7cyclictest28418-21apt-get19:20:113
2386299193,9cyclictest30731-21ssh23:00:133
2386299193,13cyclictest29821-21wc00:35:253
2386299192,7cyclictest3393-21kagekitd22:40:013
2386299192,15cyclictest4717-21cron19:40:013
23862991912,5cyclictest10025-21ssh23:08:463
23862991911,6cyclictest18765-21sed20:10:193
23862991911,6cyclictest13649-21date20:00:003
23862991911,5cyclictest636-21runrttasks23:45:263
2386299191,15cyclictest21298-21awk20:15:243
23862991910,6cyclictest624-21gdbus22:30:003
2386099199,7cyclictest278-21systemd-journal20:00:002
2386099198,9cyclictest0-21swapper/221:47:202
2386099192,12cyclictest13024-21apt-get23:35:132
2386099192,11cyclictest694-21apt-get23:50:132
2386099191,7cyclictest19913-21cpuspeed_turbos23:40:132
23860991914,3cyclictest28924-21gltestperf19:20:152
23860991914,3cyclictest25892-21sh00:33:002
23860991914,3cyclictest19829-21sh22:27:532
23860991914,3cyclictest1852-21sh23:25:562
23860991912,5cyclictest28097-21unixbench_singl20:30:262
23860991912,5cyclictest26976-21tune2fs23:45:172
23860991912,5cyclictest16419-21find00:25:262
23860991912,4cyclictest21492-21kernelversion00:05:182
23860991911,6cyclictest3314-21expr20:50:142
23860991911,6cyclictest23965-21cat19:10:132
23860991911,6cyclictest23965-21cat19:10:122
23860991911,6cyclictest12066-21df23:10:152
23860991911,5cyclictest52950irq/141-enp3s0-22:50:122
23860991911,5cyclictest11349-21diskmemload22:24:342
2385999192,12cyclictest3016-21apt-get20:50:121
23859991914,3cyclictest31966-21sh00:13:201
23859991913,4cyclictest5814-21sh21:29:541
23859991913,4cyclictest31519-21sh22:12:201
23859991913,4cyclictest23648-21sh23:18:591
23859991913,4cyclictest18940-21sh22:26:591
23859991913,3cyclictest19204-21sh00:03:541
23859991912,4cyclictest9307-21unixbench_multi21:55:261
23859991911,6cyclictest3921-21seq00:16:001
23859991911,6cyclictest32000-21seq00:37:271
23859991911,6cyclictest25052-21ssh23:20:091
23859991911,6cyclictest22056-21sed22:05:211
23859991911,6cyclictest21916-21seq20:17:581
23859991911,6cyclictest18345-21seq22:02:361
23859991911,6cyclictest15140-21sed20:00:251
23859991911,6cyclictest15140-21sed20:00:251
23859991911,5cyclictest11608-21seq21:10:031
23859991910,7cyclictest19624-21date21:40:011
23859991910,7cyclictest19624-21date21:40:011
23859991910,6cyclictest11956-21df_inode19:55:151
189862190,2chrt0-21swapper/120:10:221
2386299188,6cyclictest52750irq/139-enp3s0-22:00:133
2386299182,11cyclictest11792-21apt-get21:10:143
23862991812,4cyclictest27592-21sh21:21:003
23862991812,4cyclictest14619-21sed00:00:173
23862991812,3cyclictest25671-21gltestperf23:20:163
23862991812,3cyclictest0-21swapper/322:50:043
23862991811,5cyclictest34-21ksoftirqd/321:00:123
23862991811,5cyclictest17279-21sed20:05:253
23862991810,6cyclictest7177-21memory23:30:193
23862991810,6cyclictest32325-21expr21:25:153
23862991810,6cyclictest14604-21sed20:00:193
23862991810,6cyclictest14604-21sed20:00:183
23862991810,5cyclictest31954-21cat19:25:243
2386099188,7cyclictest396-21latency21:25:202
2386099186,5cyclictest22371-21ssh21:17:332
2386099182,11cyclictest446-21apt-get19:30:122
23860991813,3cyclictest8341-21sh21:30:412
23860991813,3cyclictest6310-21sh23:30:112
23860991813,3cyclictest28330-21grep00:10:192
23860991813,3cyclictest15550-21timerwakeupswit00:00:252
23860991812,4cyclictest3505-21kernelversion19:35:192
23860991812,3cyclictest7235-21sh23:55:072
23860991811,5cyclictest31666-21ntpq19:25:212
23860991811,5cyclictest29196-21expr20:35:122
23860991811,5cyclictest14034-21expr20:00:132
23860991811,5cyclictest14034-21expr20:00:132
23860991811,5cyclictest12821-21ntpq21:10:212
23860991810,6cyclictest24999-21ntp_kernel_err22:55:212
23860991810,6cyclictest22518-21ls20:20:132
23860991810,5cyclictest8354-21switchtime19:45:252
23860991810,5cyclictest7325-21cpu21:00:142
2385999189,8cyclictest22-21ksoftirqd/122:25:001
2385999189,6cyclictest560-21dbus-daemon20:45:001
2385999188,8cyclictest8282-21cut21:00:231
2385999181,8cyclictest3874-21ssh23:51:551
23859991813,3cyclictest31188-21sh22:35:491
23859991813,3cyclictest12631-21sh21:34:571
23859991812,4cyclictest6943-21sh22:18:291
23859991812,4cyclictest23557-21sh21:42:241
23859991812,4cyclictest19569-21sh23:40:011
23859991812,4cyclictest16997-21sh23:37:411
23859991812,4cyclictest13580-40ogrotate00:00:001
23859991811,5cyclictest53050irq/142-enp3s0-21:20:161
23859991811,5cyclictest23179-21ls00:30:241
23859991811,5cyclictest15211-21expr00:25:151
23859991810,6cyclictest278-21systemd-journal20:40:001
23859991810,6cyclictest23575-21sed20:20:241
23859991810,6cyclictest19995-21sed21:15:241
2386299179,6cyclictest9630-21python300:20:223
2386299179,5cyclictest22346-21sort20:20:113
2386299179,5cyclictest11349-21diskmemload00:34:563
2386299175,4cyclictest16768-21ssh21:37:213
2386299175,4cyclictest16768-21ssh21:37:203
2386299172,5cyclictest7336-21apt-get23:55:133
2386299171,7cyclictest31015-21ssh22:35:383
2386299171,14cyclictest0-21swapper/320:30:123
23862991710,5cyclictest8125-21grep21:55:163
23862991710,5cyclictest5864-21sed20:55:203
23862991710,5cyclictest20105-21cat21:40:143
23862991710,5cyclictest1406-21cut21:50:153
23862991710,5cyclictest11393-21cut21:10:003
2386099179,6cyclictest25460-21pluginstate20:25:242
2386099179,6cyclictest18023-21date20:10:012
2386099177,7cyclictest20480-21expr20:15:152
23860991712,3cyclictest8715-21ntp_offset21:55:212
2385999178,7cyclictest8811-21seq19:47:211
2385999178,7cyclictest560-21dbus-daemon21:10:001
23859991712,3cyclictest5130-21/usr/sbin/munin20:55:151
23859991710,5cyclictest2922-21cut20:50:001
23859991710,5cyclictest26276-21seq20:28:431
23859991710,5cyclictest17733-21seq20:09:121
2386299168,6cyclictest28969-21/usr/sbin/munin20:35:203
23862991610,4cyclictest537-21cpuspeed_turbos19:30:133
23862991610,4cyclictest10502-21sed19:50:213
2386099159,4cyclictest9747-21gltestperf19:50:142
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional