You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-12-01 - 00:45

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Phytec
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackbslot5s.osadl.org (updated Sun Nov 30, 2025 12:44:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
834181996053,5cyclictest919241-21kworker/u8:3+events_unbound11:15:570
834191995337,13cyclictest845158-21munin-run07:40:002
834191995027,18cyclictest0-21swapper/210:50:522
834195994638,5cyclictest850190-21kworker/u8:0+flush-179:9608:00:233
834191994635,8cyclictest852200-21sendmail08:00:012
834191994513,3cyclictest0-21swapper/210:35:162
834191994416,3cyclictest0-21swapper/209:50:122
834185994428,11cyclictest231ktimers/111:40:151
834195994335,5cyclictest886310-21cron09:40:003
834191994318,20cyclictest0-21swapper/212:25:012
834191994313,4cyclictest0-21swapper/207:40:202
834191994312,6cyclictest0-21swapper/211:50:202
834191994312,6cyclictest0-21swapper/211:50:192
834191994312,3cyclictest0-21swapper/210:59:592
834181994336,4cyclictest857316-21munin-plugin-st08:15:000
834181994335,6cyclictest919025-21kworker/u8:0+events_unbound11:35:590
834195994232,6cyclictest907035-21cat10:40:013
834191994234,4cyclictest0-21swapper/208:10:242
834191994215,3cyclictest0-21swapper/208:50:242
834191994213,4cyclictest0-21swapper/209:45:162
834191994211,10cyclictest879474-21cron09:20:002
834191994211,10cyclictest879474-21cron09:19:592
834191994014,7cyclictest876132-21cut09:10:012
834191994014,3cyclictest0-21swapper/208:35:212
834191994010,8cyclictest947844-21grep12:35:252
834191994010,8cyclictest947844-21grep12:35:242
834185994034,4cyclictest146-21jbd2/mmcblk1p2-808:10:241
834181994010,5cyclictest131rcu_preempt10:55:130
834191993914,3cyclictest0-21swapper/208:30:122
83418199397,4cyclictest131rcu_preempt11:05:130
834181993924,6cyclictest944902-21kworker/u8:0+events_unbound12:31:530
834191993815,3cyclictest311ktimers/207:50:132
834191993813,5cyclictest0-21swapper/212:25:262
834185993827,7cyclictest941762-21sendmail12:20:001
834185993827,6cyclictest0-21swapper/110:44:591
834181993833,3cyclictest0-21swapper/009:05:150
834181993831,4cyclictest872658-21cron09:00:010
834191993726,7cyclictest0-21swapper/209:00:262
83419199371,4cyclictest131rcu_preempt11:55:132
834191993713,5cyclictest0-21swapper/210:00:212
83419199371,12cyclictest919347-21cron11:15:012
834185993730,4cyclictest860079-21systemd08:20:251
834185993726,6cyclictest0-21swapper/111:50:241
834185993726,6cyclictest0-21swapper/111:50:241
834185993726,6cyclictest0-21swapper/109:45:241
834181993730,4cyclictest852195-21sh08:00:010
834191993625,7cyclictest0-21swapper/209:45:012
834191993615,3cyclictest0-21swapper/208:55:232
834191993612,5cyclictest311ktimers/212:00:142
834191993611,3cyclictest0-21swapper/211:05:002
834185993626,5cyclictest0-21swapper/107:20:221
834185993623,8cyclictest0-21swapper/112:40:001
834185993623,8cyclictest0-21swapper/112:40:001
834181993629,5cyclictest887818-21kworker/u8:1+events_unbound09:50:430
834181993624,7cyclictest887608-21grep09:40:240
834195993526,5cyclictest0-21swapper/308:25:003
834195993524,8cyclictest900170-21sendmail10:20:013
834185993528,5cyclictest865644-21kworker/u8:1+events_unbound08:40:271
834185993527,4cyclictest0-21swapper/108:35:161
834185993526,5cyclictest0-21swapper/112:05:001
834185993526,4cyclictest0-21swapper/107:35:271
834181993529,4cyclictest850190-21kworker/u8:0+events_unbound07:51:420
834181993526,4cyclictest0-21swapper/012:16:050
834191993426,4cyclictest0-21swapper/209:35:232
834191993425,5cyclictest0-21swapper/210:20:192
834185993425,5cyclictest853476-21grep08:00:261
834185993423,6cyclictest0-21swapper/110:20:271
834185993423,6cyclictest0-21swapper/108:55:261
834185993422,7cyclictest0-21swapper/109:09:591
834185993422,6cyclictest862167-21kworker/u8:2+events_unbound08:52:161
834181993425,5cyclictest841330-21grep07:25:270
834195993326,4cyclictest0-21swapper/312:25:273
834195993324,6cyclictest215-21dbus-daemon08:40:273
834195993324,6cyclictest0-21swapper/312:15:013
834195993323,5cyclictest937716-21grep12:05:263
834191993326,4cyclictest0-21swapper/212:35:002
834191993323,6cyclictest924900-21cut11:30:172
834191993321,9cyclictest311ktimers/207:30:122
834185993325,4cyclictest838138-21apt-get07:20:011
834185993322,6cyclictest0-21swapper/109:30:251
834181993326,4cyclictest0-21swapper/011:45:250
834181993325,4cyclictest0-21swapper/012:35:260
834181993325,4cyclictest0-21swapper/012:35:260
834181993314,5cyclictest808049-21kworker/u8:0+events_unbound07:20:480
834181993314,15cyclictest0-21swapper/012:15:010
834195993224,4cyclictest0-21swapper/311:45:263
834195993224,4cyclictest0-21swapper/310:50:003
834195993224,4cyclictest0-21swapper/310:49:593
834191993224,4cyclictest0-21swapper/210:40:282
834191993223,6cyclictest0-21swapper/207:20:012
834185993226,4cyclictest910813-21kworker/u8:2+events_unbound10:55:431
834185993225,4cyclictest943182-21kworker/u8:1+events_unbound12:31:091
834185993224,4cyclictest0-21swapper/110:30:281
834181993223,6cyclictest0-21swapper/010:40:000
834181993223,5cyclictest0-21swapper/012:05:000
83418199321,29cyclictest0-21swapper/009:45:130
834195993124,4cyclictest889943-21latency_hist09:50:003
834195993123,4cyclictest0-21swapper/310:50:183
834195993123,4cyclictest0-21swapper/309:00:013
834195993123,4cyclictest0-21swapper/307:35:273
834191993124,3cyclictest0-21swapper/208:00:252
834191993123,4cyclictest0-21swapper/211:20:162
834191993123,4cyclictest0-21swapper/207:45:262
834191993122,4cyclictest0-21swapper/207:25:272
834191993121,8cyclictest835913-21munin-node07:15:012
834191993121,6cyclictest0-21swapper/210:20:012
834185993122,5cyclictest0-21swapper/111:45:261
834181993124,3cyclictest0-21swapper/010:30:280
834181993113,10cyclictest131rcu_preempt07:35:140
834195993023,4cyclictest0-21swapper/309:05:263
834195993023,3cyclictest0-21swapper/309:15:183
834195993023,3cyclictest0-21swapper/309:15:183
834195993021,6cyclictest915952-21auditctl11:05:003
834195993020,7cyclictest948586-21sendmail12:40:003
834195993020,7cyclictest948586-21sendmail12:40:003
834195993019,6cyclictest0-21swapper/309:20:243
834195993019,6cyclictest0-21swapper/309:20:243
834191993022,4cyclictest0-21swapper/211:35:262
834185993023,5cyclictest870897-21kworker/u8:0+events_unbound09:20:341
834185993023,5cyclictest870897-21kworker/u8:0+events_unbound09:20:331
834185993022,3cyclictest0-21swapper/111:00:311
834185993021,7cyclictest231ktimers/112:10:141
834185993021,4cyclictest0-21swapper/111:15:561
834185993021,4cyclictest0-21swapper/109:15:331
834185993021,4cyclictest0-21swapper/109:15:331
83418199305,4cyclictest131rcu_preempt11:40:140
83418199303,5cyclictest934709-21munin-run12:00:000
834181993024,4cyclictest908871-21kworker/u8:1+events_unbound10:52:420
834181993024,4cyclictest859004-21sh08:19:590
834181993023,4cyclictest0-21swapper/011:30:270
834181993021,6cyclictest879482-21latency_hist09:20:000
834181993021,6cyclictest879482-21latency_hist09:20:000
834195992923,3cyclictest0-21swapper/308:20:003
834195992922,4cyclictest841342-21sed07:25:263
834195992921,4cyclictest0-21swapper/310:40:283
834195992921,4cyclictest0-21swapper/310:10:463
834195992917,7cyclictest0-21swapper/309:25:243
834195992916,11cyclictest834175-21cyclictest11:40:143
834185992922,5cyclictest919025-21kworker/u8:0+events_unbound11:56:021
834185992921,4cyclictest0-21swapper/107:50:301
834185992920,5cyclictest0-21swapper/111:35:171
834185992920,5cyclictest0-21swapper/109:50:161
834185992919,6cyclictest924020-21unixbench_multi11:25:251
834185992918,6cyclictest0-21swapper/108:05:181
834185992918,6cyclictest0-21swapper/108:05:171
83418199292,7cyclictest11-21ksoftirqd/007:10:130
834181992921,5cyclictest896768-21cut10:10:010
834181992920,7cyclictest855443-21kworker/u8:1+events_unbound08:20:220
834195992821,4cyclictest0-21swapper/311:10:273
834195992820,5cyclictest0-21swapper/311:55:273
834191992819,6cyclictest895681-21sort10:05:202
834185992821,5cyclictest887818-21kworker/u8:1+events_unbound09:42:431
834185992820,4cyclictest0-21swapper/110:45:281
834185992820,4cyclictest0-21swapper/110:45:281
834185992818,6cyclictest0-21swapper/109:55:181
834181992821,3cyclictest0-21swapper/010:20:270
834181992820,4cyclictest903594-21munin-plugin-st10:30:010
834181992813,4cyclictest131rcu_preempt11:29:000
834195992720,3cyclictest0-21swapper/309:55:223
834195992718,6cyclictest391ktimers/311:20:133
834195992718,5cyclictest0-21swapper/308:35:263
834195992718,5cyclictest0-21swapper/307:20:013
834191992720,3cyclictest0-21swapper/209:20:242
834191992720,3cyclictest0-21swapper/209:20:242
834191992718,4cyclictest0-21swapper/208:20:002
834185992720,4cyclictest941362-21kworker/u8:2+events_unbound12:25:001
834185992717,5cyclictest0-21swapper/110:10:011
834181992721,4cyclictest835442-21kworker/u8:2+events_unbound07:40:270
834181992720,4cyclictest13950irq/36-mmc110:20:010
834181992719,4cyclictest0-21swapper/012:25:260
83418199271,4cyclictest131rcu_preempt08:00:130
834181992712,5cyclictest131rcu_preempt10:11:010
83418199270,24cyclictest861741-21switchtime08:25:260
834195992620,3cyclictest0-21swapper/308:45:173
834195992619,4cyclictest0-21swapper/309:30:163
834191992619,3cyclictest0-21swapper/211:05:302
834185992619,3cyclictest0-21swapper/109:40:001
834185992619,3cyclictest0-21swapper/108:00:011
834185992617,5cyclictest0-21swapper/109:10:191
834185992617,5cyclictest0-21swapper/109:10:191
834185992615,7cyclictest944696-21grep12:25:271
834185992615,6cyclictest0-21swapper/110:35:281
83418199264,3cyclictest131rcu_preempt11:20:130
83418199263,21cyclictest0-21swapper/008:43:250
83418199263,19cyclictest0-21swapper/010:01:350
834181992619,5cyclictest906894-21kworker/u8:0+events_unbound10:41:200
834181992619,5cyclictest850190-21kworker/u8:0+events_unbound08:07:430
834181992619,5cyclictest850190-21kworker/u8:0+events_unbound08:07:430
834181992619,5cyclictest835442-21kworker/u8:2+events_unbound07:31:540
834181992619,5cyclictest835442-21kworker/u8:2+events_unbound07:17:190
834181992617,4cyclictest0-21swapper/009:29:590
83418199261,4cyclictest131rcu_preempt11:10:130
83418199261,12cyclictest0-21swapper/011:53:330
83418199261,12cyclictest0-21swapper/011:53:330
834195992519,4cyclictest53-21kcompactd008:10:233
834195992519,3cyclictest0-21swapper/310:25:233
834195992518,4cyclictest0-21swapper/307:50:293
834195992518,3cyclictest0-21swapper/311:55:003
834195992518,3cyclictest0-21swapper/311:55:003
834195992518,3cyclictest0-21swapper/309:40:153
834195992518,3cyclictest0-21swapper/308:05:183
834195992518,3cyclictest0-21swapper/308:05:183
834195992517,4cyclictest0-21swapper/311:30:173
834195992515,6cyclictest0-21swapper/312:20:003
834195992514,8cyclictest894160-21ntp_states10:00:243
834191992520,3cyclictest904856-21head10:30:272
834191992518,3cyclictest0-21swapper/208:05:262
834191992518,3cyclictest0-21swapper/208:05:252
834191992513,9cyclictest867125-21grep08:40:272
834185992518,5cyclictest870897-21kworker/u8:0+events_unbound09:04:541
834185992518,3cyclictest0-21swapper/111:30:261
834185992517,5cyclictest0-21swapper/111:15:011
834185992517,5cyclictest0-21swapper/110:17:011
834185992517,3cyclictest0-21swapper/110:10:131
834185992516,6cyclictest0-21swapper/112:10:011
834185992516,5cyclictest0-21swapper/108:15:281
834185992516,5cyclictest0-21swapper/107:10:241
834181992518,5cyclictest915577-21kworker/u8:1+events_unbound11:03:010
834181992518,4cyclictest0-21swapper/009:30:250
834181992517,5cyclictest870024-21ntpq08:50:210
834181992517,5cyclictest0-21swapper/009:10:260
834181992517,5cyclictest0-21swapper/009:10:250
834181992517,4cyclictest0-21swapper/009:55:270
834181992513,10cyclictest909934-21ntp_states10:45:230
834181992513,10cyclictest909934-21ntp_states10:45:230
834181992511,4cyclictest870897-21kworker/u8:0+events_unbound09:04:570
834195992416,6cyclictest391ktimers/307:40:243
834191992418,3cyclictest0-21swapper/211:45:192
834191992417,5cyclictest311ktimers/207:20:092
834191992416,4cyclictest0-21swapper/208:45:262
834185992418,4cyclictest835442-21kworker/u8:2+events_unbound07:26:141
834185992417,3cyclictest0-21swapper/111:20:181
834185992417,3cyclictest0-21swapper/108:30:021
834185992416,5cyclictest0-21swapper/109:25:191
834185992412,5cyclictest841425-21kworker/u8:0+events_unbound07:44:361
83418199241,9cyclictest11-21ksoftirqd/012:05:130
83418199241,19cyclictest0-21swapper/008:36:130
83418199241,19cyclictest0-21swapper/008:30:330
83418199241,13cyclictest0-21swapper/007:45:350
834195992317,3cyclictest0-21swapper/307:20:183
834195992316,4cyclictest0-21swapper/311:15:183
834195992315,5cyclictest0-21swapper/308:30:263
834195992315,4cyclictest0-21swapper/310:30:283
834191992314,6cyclictest938445-21sort12:10:162
83418599239,6cyclictest862167-21kworker/u8:2+events_unbound08:33:141
83418599239,6cyclictest841425-21kworker/u8:0+events_unbound07:30:101
834185992317,4cyclictest835442-21kworker/u8:2+events_unbound07:45:391
834185992315,4cyclictest0-21swapper/110:05:011
834185992314,6cyclictest903130-21grep10:25:271
83418199231,20cyclictest0-21swapper/009:35:140
834195992215,5cyclictest391ktimers/310:05:153
834195992215,4cyclictest0-21swapper/309:10:263
834195992215,4cyclictest0-21swapper/309:10:253
834195992214,5cyclictest0-21swapper/312:30:223
834195992213,6cyclictest870293-21grep08:50:243
834195992213,6cyclictest847564-21sort07:45:173
834195992212,7cyclictest0-21swapper/312:20:203
834195992211,8cyclictest913402-21cut10:55:233
834191992216,3cyclictest910824-21cron10:50:002
834191992216,3cyclictest910824-21cron10:50:002
834191992216,3cyclictest0-21swapper/211:25:162
834191992216,3cyclictest0-21swapper/210:25:172
834191992216,3cyclictest0-21swapper/209:30:232
834191992216,3cyclictest0-21swapper/208:20:212
834191992215,3cyclictest0-21swapper/209:55:252
834191992212,7cyclictest881652-21grep09:25:162
834191992212,7cyclictest861529-21ntp_states08:25:222
834191992212,6cyclictest0-21swapper/212:10:012
83418599222,12cyclictest0-21swapper/111:07:041
834185992214,4cyclictest0-21swapper/110:50:211
834185992214,4cyclictest0-21swapper/108:50:011
834181992214,5cyclictest0-21swapper/009:20:150
834181992214,5cyclictest0-21swapper/009:20:140
834181992212,8cyclictest11-21ksoftirqd/008:45:150
834195992115,3cyclictest0-21swapper/311:35:173
834195992115,3cyclictest0-21swapper/307:10:253
834195992113,4cyclictest0-21swapper/311:05:313
834195992111,7cyclictest936621-21cut12:05:013
834191992114,3cyclictest0-21swapper/211:40:322
834191992114,3cyclictest0-21swapper/210:10:162
834191992110,9cyclictest876620-21if_eth109:10:182
834191992110,9cyclictest876620-21if_eth109:10:182
834191992014,3cyclictest0-21swapper/212:15:192
834195991913,3cyclictest0-21swapper/308:25:223
83419599190,17cyclictest391ktimers/309:00:143
834195991812,3cyclictest900418-21/usr/sbin/munin10:20:153
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional