You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-18 - 18:06

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackeslot3s.osadl.org (updated Mon May 18, 2026 12:45:20)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1871083999418,37cyclictest12-21ksoftirqd/011:40:010
1871083998479,3cyclictest1877255-21diskmemload10:05:020
1871083998017,40cyclictest12-21ksoftirqd/011:35:010
1871083997569,4cyclictest150-21pr/ttyS212:30:030
187108399751,43cyclictest0-21swapper/010:09:260
187108399751,43cyclictest0-21swapper/010:09:250
1871083997020,10cyclictest12-21ksoftirqd/009:10:000
1871083996965,2cyclictest0-21swapper/011:20:020
1871096996758,7cyclictest37-21ksoftirqd/307:40:033
187108399666,29cyclictest1918766-21ssh10:35:120
187108399654,24cyclictest12-21ksoftirqd/012:20:020
1871083996318,7cyclictest12-21ksoftirqd/011:12:500
1871083996318,10cyclictest1978811-21cron12:40:010
1871083996218,39cyclictest1871580-21munin-run07:20:010
1871083996148,7cyclictest16850irq/134-800000012:05:000
1871083996128,16cyclictest1885610-21ssh09:26:570
187108399612,50cyclictest1942814-21cat11:25:030
1871083996121,31cyclictest16850irq/134-800000010:33:230
1871083996119,8cyclictest12-21ksoftirqd/011:00:010
1871083996118,36cyclictest1-21systemd12:10:000
1871083996114,8cyclictest12-21ksoftirqd/008:20:030
1871088996014,11cyclictest131rcu_preempt11:40:011
1871083996056,2cyclictest0-21swapper/008:25:010
1871083996039,15cyclictest3467602-21systemd-journal10:50:350
1871083996031,13cyclictest1912455-21ssh10:22:170
1871083996011,10cyclictest12-21ksoftirqd/008:05:000
1871083996010,14cyclictest12-21ksoftirqd/011:48:570
187108399592,41cyclictest1951524-21ssh11:43:260
1871083995917,7cyclictest12-21ksoftirqd/012:34:190
1871083995915,9cyclictest12-21ksoftirqd/009:59:470
187108399586,17cyclictest1923643-21sh10:45:080
1871083995815,9cyclictest12-21ksoftirqd/009:40:000
1871083995813,9cyclictest12-21ksoftirqd/010:15:050
1871096995722,25cyclictest37-21ksoftirqd/312:16:363
1871092995747,7cyclictest1862362-21kworker/2:011:42:192
1871092995747,6cyclictest1862362-21kworker/2:012:05:172
1871092995741,7cyclictest1862362-21kworker/2:012:30:262
187108399578,10cyclictest12-21ksoftirqd/011:30:010
1871083995740,9cyclictest1875690-21sendmail08:39:590
187108399572,19cyclictest1932565-21ssh11:04:060
1871092995641,6cyclictest1862362-21kworker/2:012:23:182
1871092995615,9cyclictest131rcu_preempt10:20:032
187108399562,28cyclictest1871118-21latency_hist07:10:020
1871083995613,7cyclictest12-21ksoftirqd/011:07:200
1871083995613,7cyclictest12-21ksoftirqd/010:25:230
1871083995610,8cyclictest12-21ksoftirqd/010:41:100
1871092995546,5cyclictest1862362-21kworker/2:011:58:362
1871092995539,7cyclictest1862362-21kworker/2:010:10:382
1871092995538,8cyclictest1874707-21kworker/2:311:30:262
187108899556,6cyclictest131rcu_preempt12:09:441
187108399558,12cyclictest12-21ksoftirqd/011:53:370
1871083995541,9cyclictest1899048-21munin-plugin-st09:55:000
1871083995527,12cyclictest1964823-21ssh12:10:330
1871083995512,7cyclictest12-21ksoftirqd/009:15:030
1871092995446,5cyclictest1862362-21kworker/2:007:45:012
1871092995445,6cyclictest1862362-21kworker/2:009:10:282
1871092995445,5cyclictest1862362-21kworker/2:010:39:422
187108899541,5cyclictest131rcu_preempt11:01:471
187108399547,18cyclictest1969569-21ssh12:20:260
187108399547,16cyclictest1884682-21cat09:25:020
1871083995442,6cyclictest16950irq/224-800000007:55:010
1871083995442,6cyclictest16950irq/224-800000007:55:000
1871083995424,14cyclictest1906654-21ssh10:10:080
1871083995415,7cyclictest12-21ksoftirqd/009:41:100
1871092995345,5cyclictest1862362-21kworker/2:012:17:362
1871092995345,5cyclictest1862362-21kworker/2:011:54:442
1871092995345,5cyclictest1862362-21kworker/2:007:40:022
1871092995344,7cyclictest1862362-21kworker/2:008:40:002
1871092995344,5cyclictest1874707-21kworker/2:311:29:182
1871092995344,5cyclictest1862362-21kworker/2:010:01:062
1871092995343,7cyclictest1874707-21kworker/2:311:11:202
187108899534,5cyclictest131rcu_preempt09:20:001
187108399538,5cyclictest131rcu_preempt09:18:510
187108399533,44cyclictest1876763-21unin-run09:00:010
1871092995244,5cyclictest1874707-21kworker/2:311:19:282
1871092995244,5cyclictest1874707-21kworker/2:311:07:202
1871092995244,5cyclictest1874707-21kworker/2:309:59:142
1871092995244,5cyclictest1874707-21kworker/2:309:39:032
1871092995244,5cyclictest1874707-21kworker/2:309:30:232
1871092995243,6cyclictest1862362-21kworker/2:011:46:092
1871092995243,6cyclictest1862362-21kworker/2:010:42:462
1871092995243,6cyclictest1862362-21kworker/2:010:26:012
1871092995243,5cyclictest1862362-21kworker/2:012:26:302
1871092995242,7cyclictest1862362-21kworker/2:007:58:562
1871092995241,7cyclictest1862362-21kworker/2:010:07:432
1871092995241,7cyclictest1862362-21kworker/2:010:07:422
1871092995238,5cyclictest1874707-21kworker/2:309:40:032
1871088995235,13cyclictest3467602-21systemd-journal12:15:001
1871088995213,6cyclictest131rcu_preempt11:05:081
1871083995226,11cyclictest1896218-21ssh09:48:570
1871083995226,11cyclictest1896218-21ssh09:48:570
1871083995214,10cyclictest0-21swapper/009:32:060
187109299518,39cyclictest291rcuc/208:20:022
1871092995144,4cyclictest1874707-21kworker/2:309:17:452
1871092995143,5cyclictest1874707-21kworker/2:311:20:162
1871092995143,5cyclictest1874707-21kworker/2:309:45:082
1871092995143,5cyclictest1874707-21kworker/2:309:45:072
1871092995143,5cyclictest1874707-21kworker/2:309:22:522
1871092995143,5cyclictest1862362-21kworker/2:012:10:252
1871092995143,5cyclictest1862362-21kworker/2:010:33:302
1871092995143,5cyclictest1862362-21kworker/2:010:23:262
1871092995142,6cyclictest1862362-21kworker/2:007:17:562
1871088995140,7cyclictest1876709-21cron09:00:001
1871088995114,9cyclictest131rcu_preempt10:57:361
1871088995112,9cyclictest131rcu_preempt10:36:051
1871088995111,6cyclictest131rcu_preempt11:30:261
1871083995112,7cyclictest12-21ksoftirqd/008:35:030
1871096995033,13cyclictest3467602-21systemd-journal10:35:013
1871096995032,12cyclictest275-21dbus-daemon10:15:013
1871092995043,4cyclictest1874707-21kworker/2:309:51:092
1871092995042,5cyclictest1862362-21kworker/2:012:00:162
1871092995042,5cyclictest1862362-21kworker/2:011:02:212
1871092995042,5cyclictest1862362-21kworker/2:010:55:292
1871092995042,5cyclictest1862362-21kworker/2:010:51:152
1871092995042,5cyclictest1862362-21kworker/2:008:42:572
1871092995041,6cyclictest1862362-21kworker/2:010:47:512
1871088995031,12cyclictest23-21ksoftirqd/110:35:011
1871088995019,20cyclictest23-21ksoftirqd/109:29:211
1871088995013,9cyclictest23-21ksoftirqd/109:51:071
1871083995019,28cyclictest1957696-21sshd11:55:570
1871092994941,5cyclictest1874707-21kworker/2:311:36:432
1871092994941,5cyclictest1874707-21kworker/2:309:27:482
1871092994936,6cyclictest1862362-21kworker/2:012:39:292
187108899498,6cyclictest131rcu_preempt10:23:471
1871083994930,10cyclictest309-21in:imuxsock08:14:590
1871096994836,9cyclictest1-21systemd12:40:003
1871096994828,12cyclictest1926442-21rm10:51:123
1871092994840,5cyclictest1862362-21kworker/2:007:55:012
1871092994840,5cyclictest1862362-21kworker/2:007:55:002
1871088994832,12cyclictest3467602-21systemd-journal08:20:001
1871088994821,7cyclictest23-21ksoftirqd/110:10:021
1871088994821,7cyclictest23-21ksoftirqd/110:10:021
1871088994816,7cyclictest131rcu_preempt09:50:011
1871088994816,7cyclictest131rcu_preempt09:50:001
187108899481,5cyclictest131rcu_preempt09:57:061
187108399484,13cyclictest1875271-21sed08:30:010
1871083994814,5cyclictest12-21ksoftirqd/008:00:020
1871092994739,5cyclictest1862362-21kworker/2:008:30:042
1871092994739,5cyclictest1862362-21kworker/2:007:30:032
187108899473,5cyclictest131rcu_preempt11:54:581
1871088994719,19cyclictest23-21ksoftirqd/110:15:171
1871088994710,22cyclictest23-21ksoftirqd/107:35:011
187108399478,7cyclictest12-21ksoftirqd/007:45:010
187108399474,8cyclictest12-21ksoftirqd/007:45:570
1871083994731,11cyclictest1872608-21true07:40:000
187108399472,40cyclictest1871256-21sshd07:14:520
187108399471,13cyclictest1-21systemd08:10:010
187108399471,13cyclictest1-21systemd08:10:010
1871096994632,11cyclictest1895319-21sshd09:46:563
1871096994632,11cyclictest1895319-21sshd09:46:553
1871092994638,5cyclictest1862362-21kworker/2:009:10:002
1871092994638,5cyclictest1862362-21kworker/2:008:45:572
1871092994638,5cyclictest1862362-21kworker/2:008:10:432
187108899468,4cyclictest131rcu_preempt12:21:041
1871088994618,5cyclictest131rcu_preempt11:30:021
1871088994616,11cyclictest23-21ksoftirqd/108:00:011
1871088994616,11cyclictest23-21ksoftirqd/107:20:001
187108399466,6cyclictest718586-1kworker/0:2H07:35:020
1871083994613,8cyclictest0-21swapper/007:27:580
1871096994535,7cyclictest1879906-21cat09:15:023
1871096994533,9cyclictest1-21systemd12:30:013
1871096994526,15cyclictest1-21systemd09:25:023
1871092994537,5cyclictest1862362-21kworker/2:008:58:092
187108899453,5cyclictest131rcu_preempt11:14:011
1871088994535,7cyclictest23-21ksoftirqd/112:16:311
187108899453,26cyclictest1872031-21kworker/u8:007:40:031
1871088994530,11cyclictest309-21in:imuxsock12:00:561
1871096994430,11cyclictest275-21dbus-daemon12:10:003
1871096994429,12cyclictest1-21systemd09:10:013
1871096994427,12cyclictest275-21dbus-daemon10:00:023
1871096994426,14cyclictest275-21dbus-daemon08:25:013
187109699442,4cyclictest131rcu_preempt11:35:073
187109699441,3cyclictest131rcu_preempt11:55:553
187109699441,12cyclictest1924593-21ssh10:47:193
1871092994435,6cyclictest1862362-21kworker/2:008:18:192
187108899446,8cyclictest23-21ksoftirqd/109:33:251
1871088994433,8cyclictest23-21ksoftirqd/109:10:581
1871088994419,5cyclictest23-21ksoftirqd/111:19:031
1871088994411,5cyclictest131rcu_preempt12:32:131
1871096994333,7cyclictest37-21ksoftirqd/307:30:003
1871096994329,10cyclictest1-21systemd12:05:013
1871096994327,12cyclictest1-21systemd11:55:013
1871096994326,14cyclictest275-21dbus-daemon08:40:013
1871096994326,13cyclictest275-21dbus-daemon11:05:013
187109699432,5cyclictest131rcu_preempt09:45:003
1871096994325,14cyclictest1898629-21sh09:54:013
1871092994335,5cyclictest1862362-21kworker/2:008:52:192
1871092994335,5cyclictest1862362-21kworker/2:008:05:272
1871092994335,5cyclictest1862362-21kworker/2:008:05:262
187108899439,9cyclictest23-21ksoftirqd/111:44:211
1871088994334,6cyclictest23-21ksoftirqd/110:12:381
1871088994329,9cyclictest3467602-21systemd-journal08:05:011
1871088994326,13cyclictest23-21ksoftirqd/108:40:011
187108899431,5cyclictest131rcu_preempt12:36:041
1871088994315,18cyclictest131rcu_preempt09:23:431
1871088994312,8cyclictest23-21ksoftirqd/110:41:361
187108399435,5cyclictest12-21ksoftirqd/008:40:160
187109699424,3cyclictest131rcu_preempt11:08:253
187109699423,3cyclictest131rcu_preempt09:38:113
187109699421,4cyclictest131rcu_preempt12:30:333
187109699421,3cyclictest131rcu_preempt10:55:143
1871092994234,5cyclictest1862362-21kworker/2:008:28:302
1871092994234,5cyclictest1862362-21kworker/2:007:46:212
187108899429,8cyclictest23-21ksoftirqd/109:37:571
1871088994229,10cyclictest23-21ksoftirqd/110:05:011
187108899421,5cyclictest131rcu_preempt10:50:151
1871088994212,7cyclictest131rcu_preempt11:49:401
1871088994212,12cyclictest23-21ksoftirqd/107:55:011
1871088994212,12cyclictest23-21ksoftirqd/107:55:001
1871083994225,13cyclictest364-21snmpd08:47:530
187109699413,6cyclictest131rcu_preempt10:20:043
187109699413,4cyclictest131rcu_preempt10:02:423
187109699413,3cyclictest131rcu_preempt11:30:033
1871096994128,8cyclictest37-21ksoftirqd/309:00:003
1871096994127,11cyclictest1937801-21cat11:15:023
1871096994121,15cyclictest275-21dbus-daemon11:50:013
1871096994119,17cyclictest37-21ksoftirqd/307:20:573
1871092994133,5cyclictest1862362-21kworker/2:008:00:042
1871092994133,5cyclictest1862362-21kworker/2:007:20:582
187108899418,6cyclictest131rcu_preempt09:45:021
187108899418,5cyclictest131rcu_preempt09:10:031
187108899418,10cyclictest23-21ksoftirqd/108:15:011
187108899416,10cyclictest23-21ksoftirqd/110:27:241
187108899415,5cyclictest131rcu_preempt07:25:021
1871088994127,10cyclictest1-21systemd08:55:021
1871088994120,18cyclictest23-21ksoftirqd/108:24:471
1871088994115,6cyclictest131rcu_preempt11:55:581
187108399418,4cyclictest131rcu_preempt08:52:550
1871096994032,6cyclictest37-21ksoftirqd/312:13:203
1871096994026,10cyclictest37-21ksoftirqd/310:39:443
187109699402,4cyclictest131rcu_preempt10:22:243
1871096994023,13cyclictest1-21systemd11:45:013
1871096994019,13cyclictest37-21ksoftirqd/310:44:493
1871096994017,19cyclictest37-21ksoftirqd/310:07:173
1871096994017,19cyclictest37-21ksoftirqd/310:07:173
187109699401,6cyclictest131rcu_preempt07:55:013
187109699401,6cyclictest131rcu_preempt07:55:003
1871096994011,11cyclictest37-21ksoftirqd/307:50:023
187108899409,5cyclictest131rcu_preempt08:07:451
187108899409,5cyclictest131rcu_preempt08:07:441
187108899403,7cyclictest23-21ksoftirqd/112:25:131
1871088994028,8cyclictest221rcuc/111:20:171
1871088994028,8cyclictest1871259-21sshd07:14:561
1871088994013,6cyclictest131rcu_preempt10:48:121
187108399404,31cyclictest0-21swapper/009:04:240
1871096993927,10cyclictest37-21ksoftirqd/311:15:203
1871096993926,10cyclictest37-21ksoftirqd/308:20:003
1871096993925,10cyclictest1969790-21sshd12:20:523
187109699391,5cyclictest131rcu_preempt11:20:433
1871092993927,9cyclictest1872064-21cron07:30:012
1871088993928,9cyclictest23-21ksoftirqd/107:45:001
1871088993924,12cyclictest1-21systemd08:30:001
187109699384,3cyclictest131rcu_preempt08:45:013
187109699383,3cyclictest131rcu_preempt09:32:233
1871096993828,7cyclictest1876447-21sshd08:53:573
1871096993823,11cyclictest1876975-21sshd09:02:563
1871096993820,9cyclictest37-21ksoftirqd/307:19:533
1871092993829,6cyclictest1862362-21kworker/2:009:05:012
1871088993824,9cyclictest23-21ksoftirqd/108:50:001
1871096993725,9cyclictest3467602-21systemd-journal07:56:573
187109699372,3cyclictest131rcu_preempt09:19:343
187109699371,3cyclictest131rcu_preempt11:31:233
187109699371,3cyclictest131rcu_preempt10:25:083
1871096993710,19cyclictest361rcuc/309:26:323
1871092993724,9cyclictest30-21ksoftirqd/207:14:562
1871088993724,10cyclictest1-21systemd08:45:011
187108899371,11cyclictest23-21ksoftirqd/107:25:411
187108399372,31cyclictest1871794-21sshd07:23:560
187109699361,4cyclictest131rcu_preempt08:30:003
187108899367,6cyclictest131rcu_preempt09:05:021
1871096993528,4cyclictest1874560-21latency_hist08:15:013
1871096993525,7cyclictest1873863-21sshd08:03:553
1871096993521,8cyclictest37-21ksoftirqd/307:35:003
1871088993527,6cyclictest23-21ksoftirqd/108:35:021
187109699342,5cyclictest131rcu_preempt07:14:523
1871096993423,8cyclictest1874339-21cat08:10:033
1871096993417,14cyclictest275-21dbus-daemon07:45:013
1871088993423,9cyclictest23-21ksoftirqd/107:10:021
1871088993423,8cyclictest1873203-21cat07:50:011
1871096993323,7cyclictest1874133-21modprobe08:10:003
1871096993321,9cyclictest275-21dbus-daemon08:50:023
1871096993321,9cyclictest275-21dbus-daemon08:35:013
187109299311,6cyclictest0-21swapper/207:10:022
1871096992418,4cyclictest1871017-21latency_hist07:10:033
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional